Forum Discussion
Architecture for refreshes
Hello!
We are an ISV and will be utilizing a premium capacity to embed data for our external customers, and I wanted to see if someone could provide some guidance on how best to set up dataset refreshes.
We have a multi-tenant setup (many customers in a single database and their data is loaded at different times throughout the day) We have RLS in place to filter data based on the user who logs in.
Here are the approaches I came up with, and the "Cons" I'm seeing for each:
1) Have 1 dataset with all of our customer's data in it and just use incremental refresh to update it throughout the day.
Pros:
- Easier to maintain 1 dataset
- Can be more easily shared amongst other reports
Cons:
- Since customers send data it at different times throughout the day, they could potentially have to wait a long time between when they send and the next refresh
- The dataset would be pretty large overall, and might run into limitations, both size and performance
- We do have instances where customers send us bad data or for other reasons, we need to do a reload, and I don't know how a situation for 1 customer might impact the rest (can you remove import data for just 1 customer in a dataset?)
2) Create a dataset for each tenant and add a step to the end of ETL to refresh the data.
Pros:
- Have flexibility to refresh a given customers data shortly after their load completes
- If there is an issue with a refresh or dataset, only that 1 customer is impacted (isolation)
- Smaller datasets, and thus hopefully shorter refresh times
Cons:
- Could be a pain to maintain as continue to add customers
Are there other approaches I'm missing? Any recommendations to the 2 I've outlined above? (side note: we initially started down the road of Direct Query, but the performance has been so-so, and we're afraid as we have a lot of customers using the site, it might be even worse)
Thanks!
3 Replies
- v-xicai
Community Support
Hi Anonymous ,
You can go through the links to optimize your refresh performance: https://docs.microsoft.com/en-us/power-bi/refresh-data#best-practices ,
https://docs.microsoft.com/en-us/power-bi/power-bi-reports-performance ,
https://blog.pragmaticworks.com/power-bi-performance-tips-and-techniques .
Best Regards,
Amy
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
- v-xicai
Community Support
Hi Anonymous ,
Does that make sense? If so, kindly mark my answer as a solution to help others having the similar issue and close the case. If not, let me know and I'll try to help you further.
Best regards
Amy
- AnonymousNot applicable
Hi Amy,
Thanks for sending those links, they were very helpful, especially around ideas on how to performance tune my project.
I probably didn't word it well, but my question was more around what would be a better approach:
1 large dataset that contained all of our customer or 1 dataset per customer (so smaller in size but quite a few more)