Forum Discussion
Helper VFaster API updates
Yep, dataflows have been in in General Availability since April 2019 and it's generally considered a best practice to at least stage, and often transform, your source data in them, upstream of datasets. This gives you separation of concerns with ETL in dataflows and data modeling in datasets.
I'd do the raw API calls in one dataflow, or if there's a lot of historical data I'd partition API calls in two dataflows (historical + refreshable). I'd then input that into another dataflow where additional logic (e.g. calculated columns) is performed, and that's what I would finally ingest in the dataset in Power BI Desktop.
You don't have to do it this way, but this is my considered advice based on having been there and done that quite a few times.
Helper VI just copied the screenshots off a 2018 instructional video.
https://www.youtube.com/watch?v=veuxofp0ZIg
I do want to circumvent these constant updates. I just added a calculated column to the table, and that alone prompted another 10-minute call to the API.
Community ChampionYep, dataflows have been in in General Availability since April 2019 and it's generally considered a best practice to at least stage, and often transform, your source data in them, upstream of datasets. This gives you separation of concerns with ETL in dataflows and data modeling in datasets.
I'd do the raw API calls in one dataflow, or if there's a lot of historical data I'd partition API calls in two dataflows (historical + refreshable). I'd then input that into another dataflow where additional logic (e.g. calculated columns) is performed, and that's what I would finally ingest in the dataset in Power BI Desktop.
You don't have to do it this way, but this is my considered advice based on having been there and done that quite a few times.
- bonjourposte
So I've put my API in the service, but I'm not sure how to integrate this into my data model. It's a post request, so I have brought in the necessary address columns into the service for it to figure out the scores it needs to return.
Now when I'm back in the desktop, I have connected to the dataflow, but it returns nothing, even after I refresh in both the service and desktop.
How do I get it to populate?
- otravers
Have you saved and refreshed the dataflow? It's distinct from dataset refreshes.
- bonjourposte
Yes I had... but I just ran it again, and now it's working. Thanks.