Forum Discussion
Storing static datasets in Dataflows?
- 1 year ago
Hi 13havya
There are a few approaches you can consider for storing static datasets like your legacy payroll records in Microsoft Fabric:
1. Use a Lakehouse:
• Import the Excel files into a Lakehouse as delta tables.
• This allows you to store the data in an optimized format without needing frequent refreshes.
• You can then reference this data in your dataflows or directly in reports.
2. Create a static dataflow:
• Import the Excel files into a dedicated dataflow for static data.
• Set the refresh schedule to manual or very infrequent (e.g., monthly) to minimize refresh time.
• Use the “Enter Data” option to manually input small static datasets directly in the dataflow.
3. Use parameters in dataflows:
• Create parameters to control which data gets refreshed.
• Set up logic to skip refreshing the static historical data during regular refreshes.
4. Leverage incremental refresh:
• If your data has date columns, use incremental refresh to only update new data.
• Configure the refresh policy to exclude the date range of your static historical data.Please accept this solution and give kudos if this is helpful
Thanks
Hi 13havya
There are a few approaches you can consider for storing static datasets like your legacy payroll records in Microsoft Fabric:
1. Use a Lakehouse:
• Import the Excel files into a Lakehouse as delta tables.
• This allows you to store the data in an optimized format without needing frequent refreshes.
• You can then reference this data in your dataflows or directly in reports.
2. Create a static dataflow:
• Import the Excel files into a dedicated dataflow for static data.
• Set the refresh schedule to manual or very infrequent (e.g., monthly) to minimize refresh time.
• Use the “Enter Data” option to manually input small static datasets directly in the dataflow.
3. Use parameters in dataflows:
• Create parameters to control which data gets refreshed.
• Set up logic to skip refreshing the static historical data during regular refreshes.
4. Leverage incremental refresh:
• If your data has date columns, use incremental refresh to only update new data.
• Configure the refresh policy to exclude the date range of your static historical data.
Please accept this solution and give kudos if this is helpful
Thanks
- 13havya1 year agoFrequent Visitor
Thanks nilendraFabric