Forum Discussion

RokuCap's avatar
RokuCap
Icon for Helper I rankHelper I
2 years ago
Solved

Creating a Web Query to scrape data from multiple webpages

I was following along to an exercise to 'Web scrape with Power Query' (Link). It involves using a formula in Power Query to import 'Major League Base Detailed Standings' table data from the 'baseball...
Need Help
  • PhilipTreacy's avatar
    PhilipTreacy
    2 years ago

    Hi RokuCap 

     

    Let me check, give me 10 minutes.  A quick look at the web page structure shows me that the actual table of data you want is in HTML comments, meaning the web browser wouldn't display it.

     

    So the fact we are seeing it means there's something else going on like maybe JavaScript is reading the table and rendering it to screen.  Unusual.

     

    In the meantime I've created this example PBIX file  that loads the data you want via a function.

     

    Phil

     

PhilipTreacy's avatar
PhilipTreacy
Icon for Super User rankSuper User
2 years ago

Hi RokuCap 

 

It doesn't look like you can use the 'old' way of referring to the table because as I said, the table you want isn't visible to the PQ when loaded using Web.Page(Web.Contents()) - what it sees is this

 

 

So the 7th table doesn't exist - hence the error you are seeing.

 

The reason the table you want is 'hidden' is because it's in a section of HTML comments.

 

However by using the Web.BrowserContents function in the query I created, PQ can parse the the web page code to find the table.

 

Regards

 

Phil