Forum Discussion
URL Scraping function to be limited to certain cells
Hello everyone,
i have a table with 3 columns looking like this:
trackingurl | Order Number | Shipment Status
This table gathers the above columns from another "master" table that contains many more columns and several rows for each order number (one row per item in each order).
I created the function below so that it goes through every row looking for a URL, scrapes it and delivers the result to column "Shipment Status".
let GetResults=(trackingurl) =>
let
Origine = Web.Page(Web.Contents(trackingurl)),
Data0 = Origine{0}[Data],
Children = Data0{0}[Children],
Children1 = Children{1}[Children],
Children2 = Children1{1}[Children],
Children3 = Children2{0}[Children],
Children4 = Children3{0}[Children],
Children5 = Children4{2}[Children],
Children6 = Children5{0}[Children],
Children7 = Children6{0}[Children],
Children8 = Children7{0}[Children],
Children9 = Children8{0}[Children],
#"Rinominate colonne" = Table.RenameColumns(Children9,{{"Text", "Shipment Status"}})
in
#"Rinominate colonne"
in GetResults
What i wish to achieve, is for the function to only scrape the URLs that correspond to any "Shipment Status" different from "Delivered" in order to avoid going through every single row on every update.
Can anyone help with this?
Thank so much in advance
Paolo
Power BI has no memory. It will always start from scratch. If you need to keep track of prior runs then you need to do that before the data hits Power BI.
4 Replies
- lbendlin
Super User
Power BI has no memory. It will always start from scratch. If you need to keep track of prior runs then you need to do that before the data hits Power BI.
- paulrosesNew Member
Yes, i know that it wouldn't be possible to modify PowerBI's logic, but i hoped to find a "thinking out of the box" solution.
Maybe an extra table where "delivered" rows are moved to hence removed from the table where the function scrapes URLs?
- lbendlin
Super User
I would think that "you need to do that before the data hits Power BI" is pretty much the same as "thinking out of the box" ?