Forum Discussion

SebbyP333's avatar
SebbyP333
Helper I
4 years ago
Solved

Getting additional table pages from Website

Hello Community, I am using the get data from web and there is a table with economic PMI data.  On the website you press the get more button and the website loads additional rows in the table.  Usin...
General Comment
Need Help
Show and Tell
Tips and Tricks
  • v-chenwuz-msft's avatar
    v-chenwuz-msft
    4 years ago

    Hi SebbyP333 ,

     

    This is a dynamic web page where all the data is not fully placed in the source code of the web page. When you click the show more, the browser sent a new request to the web service, and the service response an new html or some data, then the browser load the data via javascript and render it, so that you can see more.

    Power query can sent many request to the web service but can't run javascript. 

    There two ways to get all the data.

    1 Use python script. Get data by python spiders. then run python script by power query. Please note the relevant data security regulations and whether crawling of this type of data is permitted.

    2 Get the data via API which you should connect the IT of this web.

     

    Best Regards

    Community Support Team _ chenwu zhu

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

SebbyP333's avatar
SebbyP333
Helper I
4 years ago

Yes, thats right it is the show more data i would like to get.  Is the get data only limited to what is visible from the initial url page?

  • v-chenwuz-msft's avatar
    v-chenwuz-msft
    Community Support
    4 years ago

    Hi SebbyP333 ,

     

    This is a dynamic web page where all the data is not fully placed in the source code of the web page. When you click the show more, the browser sent a new request to the web service, and the service response an new html or some data, then the browser load the data via javascript and render it, so that you can see more.

    Power query can sent many request to the web service but can't run javascript. 

    There two ways to get all the data.

    1 Use python script. Get data by python spiders. then run python script by power query. Please note the relevant data security regulations and whether crawling of this type of data is permitted.

    2 Get the data via API which you should connect the IT of this web.

     

    Best Regards

    Community Support Team _ chenwu zhu

     

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.