Forum Discussion
Storing static datasets in Dataflows?
- 1 year ago
Hi 13havya
There are a few approaches you can consider for storing static datasets like your legacy payroll records in Microsoft Fabric:
1. Use a Lakehouse:
• Import the Excel files into a Lakehouse as delta tables.
• This allows you to store the data in an optimized format without needing frequent refreshes.
• You can then reference this data in your dataflows or directly in reports.
2. Create a static dataflow:
• Import the Excel files into a dedicated dataflow for static data.
• Set the refresh schedule to manual or very infrequent (e.g., monthly) to minimize refresh time.
• Use the “Enter Data” option to manually input small static datasets directly in the dataflow.
3. Use parameters in dataflows:
• Create parameters to control which data gets refreshed.
• Set up logic to skip refreshing the static historical data during regular refreshes.
4. Leverage incremental refresh:
• If your data has date columns, use incremental refresh to only update new data.
• Configure the refresh policy to exclude the date range of your static historical data.Please accept this solution and give kudos if this is helpful
Thanks
Hi 13havya,
Could you please confirm if your query have been resolved by the solution provided by nilendraFabric? If so, please mark it as the solution. This will be helpful for other community members who have similar problems to solve it faster.
Thank you.