Forum Discussion

paulroses's avatar
paulroses
New Member
5 years ago
Solved

URL Scraping function to be limited to certain cells

Hello everyone,

 

i have a table with 3 columns looking like this:

trackingurl | Order Number | Shipment Status

This table gathers the above columns from another "master" table that contains many more columns and several rows for each order number (one row per item in each order).

 

I created the function below so that it goes through every row looking for a URL, scrapes it and delivers the result to column "Shipment Status".

 

let GetResults=(trackingurl) =>

let
Origine = Web.Page(Web.Contents(trackingurl)),
Data0 = Origine{0}[Data],
Children = Data0{0}[Children],
Children1 = Children{1}[Children],
Children2 = Children1{1}[Children],
Children3 = Children2{0}[Children],
Children4 = Children3{0}[Children],
Children5 = Children4{2}[Children],
Children6 = Children5{0}[Children],
Children7 = Children6{0}[Children],
Children8 = Children7{0}[Children],
Children9 = Children8{0}[Children],
#"Rinominate colonne" = Table.RenameColumns(Children9,{{"Text", "Shipment Status"}})
in
#"Rinominate colonne"

in GetResults

 

What i wish to achieve, is for the function to only scrape the URLs that correspond to any "Shipment Status" different from "Delivered" in order to avoid going through every single row on every update.

 

Can anyone help with this?

Thank so much in advance

Paolo

  • Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

4 Replies

  • Power BI has no memory.  It will always start from scratch.  If you need to keep track of prior runs then you need to do that before the data hits Power BI.

    • paulroses's avatar
      paulroses
      New Member

      Yes, i know that it wouldn't be possible to modify PowerBI's logic, but i hoped to find a "thinking out of the box" solution. 

      Maybe an extra table where "delivered" rows are moved to hence removed from the table where the function scrapes URLs?

      • lbendlin's avatar
        lbendlin
        Icon for Super User rankSuper User

        I would think that "you need to do that before the data hits Power BI"  is pretty much the same as  "thinking out of the box" ?