Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Don't miss out! 2025 Microsoft Fabric Community Conference, March 31 - April 2, Las Vegas, Nevada. Use code MSCUST for a $150 discount. Prices go up February 11th. Register now.

Reply
Maheshwari
Regular Visitor

Web content data extraction

Hi All iam pulling data using web url. the web url shows data for only first 100 rows in one page.if we need to get further data need to click buttons 2,3,4..done using href dopostback command.but the main url remains same for all pages.so i get only first page data.is there any way to get all page data?

5 REPLIES 5
v-stephen-msft
Community Support
Community Support

Hi @Maheshwari ,

 

Please kindly refer to the blog:

https://community.powerbi.com/t5/Community-Blog/Batch-crawling-web-page-data/ba-p/1715990

 

In the blog, the author used this web site as an example, follow the steps below

1.Analysis of the structure of the website
2.Pull out the first page of data using PowerBI
3.Set the custom function based on the page number parameter
4.Batch calls to custom functions
5.Batch change to column names

 

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

Hi @v-stephen-msft ,thanks for sharing...but the web url remains same for all page numbers..it uses some href dopostback command..even looked into page source code...not sure how to get different links for each page

Hi @Maheshwari ,

 

Can you share the web url with me?

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

mahoneypat
Microsoft Employee
Microsoft Employee

That will depend on how the web page is structured.  If you see a url that contains a page value in it, you can make a list of values and then concatenate it into the url and make a web call for each page.  If not, may not be possible.

 

Pat

 





Did I answer your question? Mark my post as a solution! Kudos are also appreciated!

To learn more about Power BI, follow me on Twitter or subscribe on YouTube.


@mahoneypa HoosierBI on YouTube


I have the same problem, how to do that?

Helpful resources

Announcements
Las Vegas 2025

Join us at the Microsoft Fabric Community Conference

March 31 - April 2, 2025, in Las Vegas, Nevada. Use code MSCUST for a $150 discount! Prices go up Feb. 11th.

Feb2025 Sticker Challenge

Join our Community Sticker Challenge 2025

If you love stickers, then you will definitely want to check out our Community Sticker Challenge!

Jan NL Carousel

Fabric Community Update - January 2025

Find out what's new and trending in the Fabric community.

Top Solution Authors