Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
Cmoore
Frequent Visitor

Web Scraping

I am new to web scraping using Power BI. I have had some successes of using the software to extract data from easily built websites and iterated over multiple pages. I am struggling with sites that are heavily coded in java where the data isn't fully displayed in a table. I am trying to connect to a website below and this is the M code;

l

et
Source = Web.BrowserContents("https://www.kirkland.com/lawyers?level=9ff51805-f16d-4d64-9794-6cff75200182"),
#"Extracted Table From Html" = Html.Table(Source, {{"Persons Name", ".person-result__name"}, each [Attributes][href]?}, [RowSelector=".person-result__main > A"]),
#"Changed Type" = Table.TransformColumnTypes(#"Extracted Table From Html",{{"Persons Name", type text}})
in
#"Changed Type"

 

I am just trying to extract the person name for each of the occurences on the page. I get the error "We Cannot covert a value of type function to type list". Would you be able to offer any guidance to what I'm doing wrong on a website like this?

2 REPLIES 2
Cmoore
Frequent Visitor

Thanks for getting back to me, much appreciated.

Daryl-Lynch-Bzy
Resident Rockstar
Resident Rockstar

Sorry @Cmoore - I don't think Power Query can help in this situation.  If you look at the result of the "Web.BrowserContents".  Show that Javascript is disabled, so you get a big text string.

DarylLynchBzy_0-1644405773922.png

The most interesting part of this string is the value="PE1...." part.  This can be read using:

 Text.FromBinary(Web.Contents("https://www.kirkland.com/lawyers?letter=A") , BinaryEncoding.Base64),

but it only returns the page structure not the lawyer data

 

DarylLynchBzy_1-1644406044200.png

 

It looks like you need to use an API to get the information.

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

PBI_APRIL_CAROUSEL1

Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors