Skip to main content
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

A new Data Days event is coming soon! This time we’re going bigger than ever. Fabric, Power BI, SQL, AI and more. Don't miss out.

Reply
Gopal2704
Frequent Visitor

Power Query Pivot stuck

‎07-11-2023 02:02 AM

Hi Everyone !
I have been trying to pivot the below data by wrt to Account_L2 and Values.
There are around 5 million such rows
But it is taking a hell lot of time and finally crashing my power BI.

The source is Dataflow.
Please help

MonthYearZONE 1ZONE 2CLUSTER 1CLUSTER 2CLUSTER 3CLUSTER 4COUNTRYFDT ClusterNBM CountriesFranchiseFranchise#2Division#1Division#2PRODUCT FAMILYPRODUCTMAIN PRODUCTEXT - SEGMENTEXT - ASSET TYPEEXT - PRODUCT FAMILY NEWINT - SEGMENTINT - ASSET TYPEINT - PRODUCT FAMILY NEWMARKET#1MARKET#2Other information #1Other information #2Account_L2INT - FDT SEGMENTRatesATT_DIVATT_EnActiviteMC CodeAccount CodeProduct CodeExtracted RatesAttributeValue
Dec2021GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGROSS MARGIN GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINES713000
Dec2021GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGROSS MARGIN GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINES-713000
Dec2021GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGROSS MARGIN GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINES0
Dec2021GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGROSS MARGIN GENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINESGENERAL MEDICINES0
Labels:
Message 1 of 3
690 Views
0
1 ACCEPTED SOLUTION
BA_Pete
Super User
Super User

‎07-11-2023 04:28 AM

Hi @Gopal2704 ,

 

Pivot/Unpivot are 'whole-table' operations i.e. Power Query must load the ENTIRE table into memory in order to perform the transformation.

 

For such large tables, you have a few options:

-1- Don't perform the pivot at all and see if it's possible to use the data in its native state e.g. by using matrix visuals etc.

-2- Perform the pivot operation at source e.g. write a view on the server that pivots the data before being brought in to Dataflow/Power Query.

-3- Use the original source (SQL Server etc.) as your query source (instead of a Dataflow) and utilise query folding to perform the operation at source.

-4- Break up/filter your table into much smaller pieces before performing the pivot/unpivot.

-5- Invest in RAM to go into your gateways/desktop/laptop, cranking them all up to 48GB-64GB+.

 

You may also find performance improvements by doing the pivot in DAX, although this will likely also be highly dependent on machine RAM availability as well.

 

Pete



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




View solution in original post

Message 2 of 3
656 Views
0
2 REPLIES 2
BA_Pete
Super User
Super User

‎07-11-2023 04:28 AM

Hi @Gopal2704 ,

 

Pivot/Unpivot are 'whole-table' operations i.e. Power Query must load the ENTIRE table into memory in order to perform the transformation.

 

For such large tables, you have a few options:

-1- Don't perform the pivot at all and see if it's possible to use the data in its native state e.g. by using matrix visuals etc.

-2- Perform the pivot operation at source e.g. write a view on the server that pivots the data before being brought in to Dataflow/Power Query.

-3- Use the original source (SQL Server etc.) as your query source (instead of a Dataflow) and utilise query folding to perform the operation at source.

-4- Break up/filter your table into much smaller pieces before performing the pivot/unpivot.

-5- Invest in RAM to go into your gateways/desktop/laptop, cranking them all up to 48GB-64GB+.

 

You may also find performance improvements by doing the pivot in DAX, although this will likely also be highly dependent on machine RAM availability as well.

 

Pete



Now accepting Kudos! If my post helped you, why not give it a thumbs-up?

Proud to be a Datanaut!




Message 2 of 3
657 Views
0
Gopal2704
Frequent Visitor
In response to BA_Pete

‎07-17-2023 12:33 AM

Thank you for providing a detailed solution Pete !!

Message 3 of 3
639 Views

Helpful resources

Announcements
May Power BI Update Carousel

Power BI Monthly Update - May 2026

Check out the May 2026 Power BI update to learn about new features.

Fabric SQL PBI Data Days

Data Days 2026 coming soon!

Sign up to receive a private message when registration opens and key events begin.

New to Fabric survey Carousel

New to Fabric Survey

If you have recently started exploring Fabric, we'd love to hear how it's going. Your feedback can help with product improvements.

Power BI DataViz World Championships carousel

Power BI DataViz World Championships - June 2026

A new Power BI DataViz World Championship is coming this June! Don't miss out on submitting your entry.

View All
Top Solution Authors
View All