Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
7 years ago
Solved

Website scraping advice

I'm trying to scrape a website with a pretty simple HTML table, but it uses Javascript for pagination and I can only get the first 25 results when using the web connector. I've tried using  [WaitFo...
Need Help
Tutorial Requests
  • v-yuta-msft's avatar
    v-yuta-msft
    7 years ago

    Anonymous ,

     

    I would suggest you to use python script in power bi to scrapy the website. About how to configure python environment and implement python script in power bi desktop, I would suggest you to refer to doc below:

    https://docs.microsoft.com/en-us/power-bi/desktop-python-scripts

     

    Community Support Team _ Jimmy Tao

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • Anonymous's avatar
    Anonymous
    Not applicable
    7 years ago
    It started as promising, but unfortunately I can't get any parameters from the url as it doesn't produce any when you navigate through the pages... Tricky!
    • v-yuta-msft's avatar
      v-yuta-msft
      Community Support
      7 years ago

      Anonymous ,

       

      I would suggest you to use python script in power bi to scrapy the website. About how to configure python environment and implement python script in power bi desktop, I would suggest you to refer to doc below:

      https://docs.microsoft.com/en-us/power-bi/desktop-python-scripts

       

      Community Support Team _ Jimmy Tao

      If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.