Skip to main content
Showing results for 
Search instead for 
Did you mean: 

Find everything you need to get certified on Fabric—skills challenges, live sessions, exam prep, role guidance, and more. Get started

New Member

Need help with bottleneck in my algorithm

As a programmer, I've encountered a problem with the data flow algorithm in my code. I have a for loop that processes a large amount of data, but it seems to slow down significantly as the loop progresses. I suspect this is due to a bottleneck in my algorithm, where a lack of resources or inefficiency in my code limits the processing time.

Here's an example of the problematic code:

for i in range(10000000):
  # process data
  data = fetch_data()
  result = process_data(data)

I need to find a way to optimize the processing time so that my algorithm can handle a large amount of data efficiently. I've tried reading this site about the dataflow but I need more support can someone help me?


Helpful resources

Europe Fabric Conference

Europe’s largest Microsoft Fabric Community Conference

Join the community in Stockholm for expert Microsoft Fabric learning including a very exciting keynote from Arun Ulag, Corporate Vice President, Azure Data.

Power BI Carousel June 2024

Power BI Monthly Update - June 2024

Check out the June 2024 Power BI update to learn about new features.

RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

Top Solution Authors
Top Kudoed Authors