Forum Discussion

Veles's avatar
Veles
Advocate V
7 years ago

Web scrape from javascript loaded pages

We're trying to use the new Web query to scrape data from a website. However when you browse the site it doesn't have all items listed on one page. It appears that it loads the next page of items when you scroll to the bottom using what I assume is Javascript.

 

When we query the web page using Power BI it will just load the first set of items. As there's no page number in the URL we can't just run a parameter to iterate over all the pages.

 

Is there any way we can force load the whole lot or perhaps see the underlying URL of the page number?