Forum Discussion
Web query shows empty
Hi guys,
I've been trying to scrape data from a company's website, something very simple. I basically want to retrieve the headlines, url and publication date from their news release.
I've already done similar queries with Google news and other news websites. But for this one I can't seem to figure out the reason why it won't retrieve anything.
Below is the detail of my query. The output is an empty table. Thanks for any help you can bring.
let
Source = Web.Contents("https://ir.tevapharm.com/news-and-events/press-releases/default.aspx"),
#"Extracted Table From Html" = Html.Table(Source, {{"Column0","a[href^=""/news-""]", each [Attributes][href]}, {"Column1", ".module-news .module_headline"}, {"Column2", ".module-news .module_date-time"}}, [RowSelector=".module-news .module_item-wrap"]),
#"Replaced Value" = Table.ReplaceValue(#"Extracted Table From Html","/news-","https://www.ir.tevapharm.com/news-",Replacer.ReplaceText,{"Column0"}),
#"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Column0", type text}, {"Column1", type text}, {"Column2", type date}}),
#"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Column0", "URL"}, {"Column1", "Headlines"}, {"Column2", "Date"}})
in
#"Renamed Columns"
8 Replies
Hi myriam_ouan ,
In your Source Step, try changing Web.Contents("https....) to Web.BrowserContents("https....).
Pete
Hi Pete,
thanks for your answer. I tried but I have an error message : can't connect to [url]
Hi myriam_ouan ,
I don't know what to suggest I'm afraid - I can't recreate this issue.
If I take your original query and make the Source change, I get all the data just fine.
Try pasting this into a new blank query, just in case you have a typo:
let Source = Web.BrowserContents("https://ir.tevapharm.com/news-and-events/press-releases/default.aspx"), #"Extracted Table From Html" = Html.Table(Source, {{"Column0","a[href^=""/news-""]", each [Attributes][href]}, {"Column1", ".module-news .module_headline"}, {"Column2", ".module-news .module_date-time"}}, [RowSelector=".module-news .module_item-wrap"]), #"Replaced Value" = Table.ReplaceValue(#"Extracted Table From Html","/news-","https://www.ir.tevapharm.com/news-",Replacer.ReplaceText,{"Column0"}), #"Changed Type" = Table.TransformColumnTypes(#"Replaced Value",{{"Column0", type text}, {"Column1", type text}, {"Column2", type date}}), #"Renamed Columns" = Table.RenameColumns(#"Changed Type",{{"Column0", "URL"}, {"Column1", "Headlines"}, {"Column2", "Date"}}) in #"Renamed Columns"Pete
Hi Pete,
thanks for your answer. I tried but I have an error message : can't connect to [url]
Hi myriam_ouan ,
Did you try to create a new blank query and then paste BA_Pete 's syntax by using
Web.BrowserContents()After edit the credential (please choose Anonymous access),I could get data succesfully as well.
Make sure your PBI is the latest version.
Best Regards,
Eyelyn Qin
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.I tried to copy and paste the full code you sent and I received the same error message. The message in French says : we could not retrieve the content from [url]. When I click "go to error" it brings me back to the source (see first screenshot). My version of power BI is May2021 : 2.93.982.0 64-bit
Hi myriam_ouan ,
It appears as thought this is a 407 error caused by failed Proxy Authentication.
I'm not an expert in this area, but it may be related to your organisation firewall and/or secure VPN proxy server authentication.
It might help to check with your organisation if they allow browser connections to that URL and, if not, to whitelist it for you.
Pete
Hi myriam_ouan ,
As I mentioned before, please always use the latest version of PBI.
Best Regards,
Eyelyn Qin
