Web query shows empty

[myriam_ouan]

Hi guys,

I've been trying to scrape data from a company's website, something very simple. I basically want to retrieve the headlines, url and publication date from their news release. 
I've already done similar queries with Google news and other news websites. But for this one I can't seem to figure out the reason why it won't retrieve anything. 

Below is the detail of my query. The output is an empty table. Thanks for any help you can bring. 

let

    Source = Web.Contents("https://ir.tevapharm.com/news-and-events/press-releases/default.aspx"),

    #"Extracted Table From Html" = Html.Table(Source, {{"Column0","a[href^=""/news-""]", each [Attributes][href]}, {"Column1", ".module-news .module_headline"}, {"Column2", ".module-news .module_date-time"}}, [RowSelector=".module-news .module_item-wrap"]),

    #"Replaced Value" = Table.ReplaceValue(#"Extracted Table From Html","/news-","https://www.ir.tevapharm.com/news-",Replacer.ReplaceText,{"Column0"}),

    #"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Column0", type text}, {"Column1", type text}, {"Column2", type date}}),

    #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Column0", "URL"}, {"Column1", "Headlines"}, {"Column2", "Date"}})

in

    #"Renamed Columns"

[Anonymous]

Hi @myriam_ouan ,

As I mentioned before, please always use the latest version of PBI.

Best Regards,
Eyelyn Qin

[myriam_ouan]

I tried to copy and paste the full code you sent and I received the same error message. The message in French says : we could not retrieve the content from [url]. When I click "go to error" it brings me back to the source (see first screenshot). My version of power BI is May2021 : 2.93.982.0 64-bit

 

 

[BA_Pete]

Hi @myriam_ouan ,

It appears as thought this is a 407 error caused by failed Proxy Authentication.

I'm not an expert in this area, but it may be related to your organisation firewall and/or secure VPN proxy server authentication.

It might help to check with your organisation if they allow browser connections to that URL and, if not, to whitelist it for you.

Pete

[Anonymous]

Hi @myriam_ouan ,

Did you try to create a new blank query and then paste @BA_Pete 's syntax by using

Web.BrowserContents()

After edit the credential (please choose Anonymous access),I could get data succesfully as well.

Make sure your PBI is the latest version.

Best Regards,
Eyelyn Qin
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[BA_Pete]

Hi @myriam_ouan ,

In your Source Step, try changing Web.Contents("https....) to Web.BrowserContents("https....).

Pete

[Syndicate_Admin]

Hi Pete, 

thanks for your answer. I tried but I have an error message : can't connect to [url]

[myriam_ouan]

Hi Pete, 

thanks for your answer. I tried but I have an error message : can't connect to [url]

[BA_Pete]

Hi @myriam_ouan ,

I don't know what to suggest I'm afraid - I can't recreate this issue.

If I take your original query and make the Source change, I get all the data just fine.

Try pasting this into a new blank query, just in case you have a typo:

let
    Source = Web.BrowserContents("https://ir.tevapharm.com/news-and-events/press-releases/default.aspx"),
    #"Extracted Table From Html" = Html.Table(Source, {{"Column0","a[href^=""/news-""]", each [Attributes][href]}, {"Column1", ".module-news .module_headline"}, {"Column2", ".module-news .module_date-time"}}, [RowSelector=".module-news .module_item-wrap"]),
    #"Replaced Value" = Table.ReplaceValue(#"Extracted Table From Html","/news-","https://www.ir.tevapharm.com/news-",Replacer.ReplaceText,{"Column0"}),
    #"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Column0", type text}, {"Column1", type text}, {"Column2", type date}}),
    #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Column0", "URL"}, {"Column1", "Headlines"}, {"Column2", "Date"}})
in
    #"Renamed Columns"

Pete