Forum Discussion
Faster API updates
- 4 years ago
Yep, dataflows have been in in General Availability since April 2019 and it's generally considered a best practice to at least stage, and often transform, your source data in them, upstream of datasets. This gives you separation of concerns with ETL in dataflows and data modeling in datasets.
I'd do the raw API calls in one dataflow, or if there's a lot of historical data I'd partition API calls in two dataflows (historical + refreshable). I'd then input that into another dataflow where additional logic (e.g. calculated columns) is performed, and that's what I would finally ingest in the dataset in Power BI Desktop.
You don't have to do it this way, but this is my considered advice based on having been there and done that quite a few times.
Sounds like you're doing all of this in Power BI Desktop, correct? If that's the case consider moving your API calls to dataflows. You might have to adapt your Power Query code so that it can work in the service but in my experience it's well worth it.