Forum Discussion

J94's avatar
J94
Advocate II
9 months ago
Solved

Extract data from website

Hi everyone, I want to extract dividend data from the site dividendhistory. However, with underlying code, I am only able to extract a few rows instead of the whole table. And this stock has sever...
Need Help
  • PhilipTreacy's avatar
    PhilipTreacy
    9 months ago

    J94 

     

    That site is loading data with JavaScript so unfortunately you can't use Power Query to get the data.

     

    PQ requires that the data can be retrieved with parameters that are supplied in the URL (or possibly in headers).  Unless the site API allows this you are stuck.

     

    You could use somethign like Selenium to get the data but that requires further programming outside PQ.

     

    There are webites that provide scraping as a service, you can search for them, but that will cost you and require integration into PQ.

     

    Phil

danextian's avatar
danextian
Super User
9 months ago

Hi J94 

I have inspected the site using the developer tool. Navigating to a page or changing the number of items creates a  post request which real API endpoint is unfortunately hidden behind a javascript. 

 

If you change the page size to 100 there will only be 3 pages so extracting the data manually shouldn't take that long.