Forum Discussion

Maheshwari's avatar
Maheshwari
Regular Visitor
4 years ago

Web content data extraction

Hi All iam pulling data using web url. the web url shows data for only first 100 rows in one page.if we need to get further data need to click buttons 2,3,4..done using href dopostback command.but th...
Need Help
Anonymous's avatar
Anonymous
Not applicable
4 years ago

Hi Maheshwari ,

 

Please kindly refer to the blog:

https://community.powerbi.com/t5/Community-Blog/Batch-crawling-web-page-data/ba-p/1715990

 

In the blog, the author used this web site as an example, follow the steps below

1.Analysis of the structure of the website
2.Pull out the first page of data using PowerBI
3.Set the custom function based on the page number parameter
4.Batch calls to custom functions
5.Batch change to column names

 

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • Maheshwari's avatar
    Maheshwari
    Regular Visitor
    4 years ago

    Hi Anonymous ,thanks for sharing...but the web url remains same for all page numbers..it uses some href dopostback command..even looked into page source code...not sure how to get different links for each page

    • Anonymous's avatar
      Anonymous
      Not applicable
      4 years ago

      Hi Maheshwari ,

       

      Can you share the web url with me?

       

      Best Regards,

      Stephen Tao

       

      If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.