Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

Reply
djwbaker
Frequent Visitor

SAP BW connector batch sizing dataflow

Hi

I have a SAP query that downloads daily to a dataset but i cant set it up successfully in a dataflow.

Its large! and the dataflow always runs out of memory. i think the dataset is successful due to the batch sizing. 50K lines at a time.

 

Does anyone know if its possible to add this option to a dataflow?

 

Dan

1 ACCEPTED SOLUTION
v-deddai1-msft
Community Support
Community Support

Hi @djwbaker ,

 

You can use incremental refresh with dataflows to resource consumption. Please refer to https://docs.microsoft.com/en-us/power-query/dataflows/incremental-refresh  and if you have a premium capacity, you can set Max memory used for dataflow in the capacity : https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-premium-workload-confi...

 

There is no option  that can eanble batch sizing or you can talking about  container Size, it is used for ETL processes. The mashup engine splits your query logic into these containers and they can process in parallel.

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Best Regards,

Dedmon Dai

View solution in original post

1 REPLY 1
v-deddai1-msft
Community Support
Community Support

Hi @djwbaker ,

 

You can use incremental refresh with dataflows to resource consumption. Please refer to https://docs.microsoft.com/en-us/power-query/dataflows/incremental-refresh  and if you have a premium capacity, you can set Max memory used for dataflow in the capacity : https://docs.microsoft.com/en-us/power-bi/transform-model/dataflows/dataflows-premium-workload-confi...

 

There is no option  that can eanble batch sizing or you can talking about  container Size, it is used for ETL processes. The mashup engine splits your query logic into these containers and they can process in parallel.

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

 

Best Regards,

Dedmon Dai

Helpful resources

Announcements
Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

PBI_Carousel_NL_June

Fabric Community Update - June 2024

Get the latest Fabric updates from Build 2024, key Skills Challenge voucher deadlines, top blogs, forum posts, and product ideas.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

Top Solution Authors