Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Enhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.

Reply
13havya
Frequent Visitor

Storing static datasets in Dataflows?

Hi,

 

Just trying to figure out what the best way is, to store static datasets in dataflows? 

We have legacy payroll records that are stored in excel files, which have a bunch of transformations applied to accomodate it to our data model. These records are no longer modified, as we have an API connector feeding into a datalake, which we use for the payroll information, but we still require the legacy reports to be connected to our datasets (to display historical data). 

As the legacy records contain over 1000 files over a period of 3 years, it is not feasible to refresh them along with other dataflows. What's the best way to store this data, to minimise/completely remove refresh time 

1 ACCEPTED SOLUTION
nilendraFabric
Super User
Super User

Hi @13havya 

 

There are a few approaches you can consider for storing static datasets like your legacy payroll records in Microsoft Fabric:
1. Use a Lakehouse:
• Import the Excel files into a Lakehouse as delta tables.
• This allows you to store the data in an optimized format without needing frequent refreshes.
• You can then reference this data in your dataflows or directly in reports.
2. Create a static dataflow:
• Import the Excel files into a dedicated dataflow for static data.
• Set the refresh schedule to manual or very infrequent (e.g., monthly) to minimize refresh time.
• Use the “Enter Data” option to manually input small static datasets directly in the dataflow.
3. Use parameters in dataflows:
• Create parameters to control which data gets refreshed.
• Set up logic to skip refreshing the static historical data during regular refreshes.
4. Leverage incremental refresh:
• If your data has date columns, use incremental refresh to only update new data.
• Configure the refresh policy to exclude the date range of your static historical data.

 

Please accept this solution and give kudos if this is helpful 

 

Thanks

View solution in original post

4 REPLIES 4
v-ssriganesh
Community Support
Community Support

Hi @13havya,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

v-ssriganesh
Community Support
Community Support

Hi @13havya,

Could you please confirm if your query have been resolved by the solution provided by @nilendraFabric? If so, please mark it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.

nilendraFabric
Super User
Super User

Hi @13havya 

 

There are a few approaches you can consider for storing static datasets like your legacy payroll records in Microsoft Fabric:
1. Use a Lakehouse:
• Import the Excel files into a Lakehouse as delta tables.
• This allows you to store the data in an optimized format without needing frequent refreshes.
• You can then reference this data in your dataflows or directly in reports.
2. Create a static dataflow:
• Import the Excel files into a dedicated dataflow for static data.
• Set the refresh schedule to manual or very infrequent (e.g., monthly) to minimize refresh time.
• Use the “Enter Data” option to manually input small static datasets directly in the dataflow.
3. Use parameters in dataflows:
• Create parameters to control which data gets refreshed.
• Set up logic to skip refreshing the static historical data during regular refreshes.
4. Leverage incremental refresh:
• If your data has date columns, use incremental refresh to only update new data.
• Configure the refresh policy to exclude the date range of your static historical data.

 

Please accept this solution and give kudos if this is helpful 

 

Thanks

Thanks @nilendraFabric 

Helpful resources

Announcements
July 2025 community update carousel

Fabric Community Update - July 2025

Find out what's new and trending in the Fabric community.

June FBC25 Carousel

Fabric Monthly Update - June 2025

Check out the June 2025 Fabric update to learn about new features.