Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago
Solved

Website scraping advice

I'm trying to scrape a website with a pretty simple HTML table, but it uses Javascript for pagination and I can only get the first 25 results when using the web connector. I've tried using  [WaitFo...
Need Help
Tutorial Requests
  • v-yuta-msft's avatar
    v-yuta-msft
    7 years ago

    Anonymous ,

     

    I would suggest you to use python script in power bi to scrapy the website. About how to configure python environment and implement python script in power bi desktop, I would suggest you to refer to doc below:

    https://docs.microsoft.com/en-us/power-bi/desktop-python-scripts

     

    Community Support Team _ Jimmy Tao

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Anonymous's avatar
Anonymous
Not applicable
7 years ago

I'm afraid my skills at this point won't allow for python scripting, so in the meanwhile I've downloaded the page as .html and I used the Text/CSV data connector to get the table in plain HTML. The downside of course is I cannot get the latest updates to my report over the internet.

v-yuta-msft's avatar
v-yuta-msft
Community Support
7 years ago

Anonymous ,

 

Power query only support simple web scrapying. If the website needs dynamic scrapying, I'm afraid power query won't work.

 

Community Support Team _ Jimmy Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.