Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Get Fabric Certified for FREE during Fabric Data Days. Don't miss your chance! Request now

Reply
TheVelvetTurtle
New Member

How best to handle large data sets and how do I best merge a dataset with a table from power query

Hi All,

Long time listener, first time caller...   

I have a report that we want to view data across multiple years.  The data tables are massive and load times are in many hours to days.  I thought a reasonable strategy was to create a data set for fixed data (years 2021, 2022) and then use an ODATA feed to bring in and manipulate 2023 to date.  I managed to get the data set created and published, and I managed to get it imported into my report as a data set.  A couple of questions:

  • Is this the best strategy for a large data set?  It is confidential information so I do not really want to create a new data lake/storage solution since the security controls are built into the ODATA source.  Any other thoughts on how to accomplish this?
  • How do I merge the dataset data with the ODATA feed data.  In the end I have 1 table from each data source I want to merge together but the dataset does not show in PowerQuery.  I am trying Union but the tables are not exactly the same so it is failing.  Currently trying to make the tables have the same column structure so this works.  Is this the best approach?  Is there another way I can append the current data from the ODATA feed to the static data in the data set?

Thank you in advance for any suggestions.  

1 REPLY 1
amitchandak
Super User
Super User

@TheVelvetTurtle , for strategy refer

https://www.btelligent.com/en/blog/power-bi-with-large-data-quantities-three-pragmatic-tips/

https://www.youtube.com/watch?v=qZOEDBedATA

Share with Power BI Enthusiasts: Full Power BI Video (20 Hours) YouTube
Microsoft Fabric Series 60+ Videos YouTube
Microsoft Fabric Hindi End to End YouTube

Helpful resources

Announcements
Fabric Data Days Carousel

Fabric Data Days

Advance your Data & AI career with 50 days of live learning, contests, hands-on challenges, study groups & certifications and more!

October Power BI Update Carousel

Power BI Monthly Update - October 2025

Check out the October 2025 Power BI update to learn about new features.

FabCon Atlanta 2026 carousel

FabCon Atlanta 2026

Join us at FabCon Atlanta, March 16-20, for the ultimate Fabric, Power BI, AI and SQL community-led event. Save $200 with code FABCOMM.

Top Solution Authors