Forum Discussion
Web scraping on dynamic websites
Hi,
I have been looking at scraping some data from a few different websites using the methods in the link below which was really helpful.
https://datachant.com/2017/03/30/web-scraping-power-bi-excel-power-query/
However, I'm trying to use the same method on a different site and I'm getting stuck! When I build the query it's only returning a small number of values (in this case I'm drilling down to the product name). I believe it's because of the way the site is built, that it only loads more new rows as you scroll down in the browser window, rather than showing a set number of values at a time.
https://groceries.asda.com/dept/chilled-food/cheese/111678
Has anyone succesfully scraped a site that behaves like this and what methods did you use?
Thanks
1 Reply
Hi bangorkeith
If the site requires interaction to load more data then you will need to be able to interact with the site to do this. I'd use VBA and something like Selenium to achieve this,
https://www.myonlinetraininghub.com/web-scraping-filling-forms
Phil
If I answered your question please mark my post as the solution.
If my answer helped solve your problem, give it a kudos by clicking on the Thumbs Up.
