Forum Discussion
Web Scraping always bringing back first 10 line items only!
Hello all,
I am trying to use the web scraping feature in Power BI to extract all open job positions from the following site: Estee Lauder Job Search - Jobs - Page 1 (elcompanies.com)
It shows the top 100 open vacancies but when ever i try to extract, it always only extracts the first 10 openings. I have also changed the page to show only 10 vacanices per page and then used a custom invoke logic to bring back the 10 openings per every page. When i do this it always shows me the same 10 job openings. Even if i enter the URL for page 2 of the vacancies, it still only brings back the first 10 vacancies from page one.
Has anyone faced anything like this before or know of a solution to get around this?
Your help would be appreciated, many thanks in advance
Hello. I am using the version from the microsoft store.
I have found a work around by bringing the data through data flow on the service which seems to have no issues in extracting all the data 😊
9 Replies
- Vijay_A_Verma
Most Valuable Professional
Not able to reproduce as I get all 100 through the above link. Are you on latest version of Power BI Desktop (Mar 2022)? See it in Help - About
hello, yes i am on March 2020 version. Very strange that you get all 100 and i only keep getting 10 rows extracted. Are you just copying the web link using the Web data source?
Are you using the link https://jobs.elcompanies.com/estee-lauder-companies/jobs?page=1&limit=100
- Vijay_A_Verma
Yes, same link. See the screenshot below to see more than 10 records
ah thats so not fair its working for you but not me 😕
Hey anychance you could explain how you solved this through a dataflow? I'm having the exact same problem with a different website.
What i did is use the web scraping as you would in desktop, copy the query and then paste into a dataflow blank query. This worked for me and showed all records, on desktop still shows on 10 records for some reason
