Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Grow your Fabric skills and prepare for the DP-600 certification exam by completing the latest Microsoft Fabric challenge.

Reply
murrayb8
Helper I
Helper I

How to improve refresh times for multiple references of one data source

Hi,

 

I have a large data set with 300+MB csv file and im trying to cut down refresh times,

what Im trying to do is delimit one of the columns, duplicate and recombine all the

delimited columns with the other associated colums still included.

E.G.

This would be the raw data:

Value    Text

100       this is an example

300       this is another example

 

after delimiting only the text column I would get the following:

Value    Text   Text1   Text2   Text3        Text4

100       this    is        an        example   null

300       this    is        one      more        example

 

and then I want to duplicate this data so I can get total values for each word used like so:

Value   Text

100      this

100      is

100      an

100     example

300     this

300     is

300    one

300    more

300    example

 

I've managed to do this by creating multiple references of the original data source, deleting corresponding columns and then recombining them but with the files being so large and some text values haveing up to 15 words it's taking a LONG amount of time to refresh each individual data source. As I only have one actual data source I presumed that the file would only have to refresh one data source but it is refreshing all of the references which is what is taking so long.

 

Can anyone help me to speed up this refresh please?

Maybe an alternative way of doing this might work better, any help is much appreciated!

 

 

2 REPLIES 2
blopez11
Resident Rockstar
Resident Rockstar

I'm not sure it is faster (processing time), but should eliminate alot of steps

In the query editor, after delimiting your text columns, select all text columns (result of your delimiting), then right click then select "Unpivot Columns".  This should give you the following in one swoop:

100      this

100      is

100      an

100     example

300     this

300     is

300    one

300    more

300    example

Anonymous
Not applicable

This is an interesting questioning and it relates to a question I want to search or post... "What is the recommended computer speed (CPU/RAM) to have to run PowerBI. 

 

Murray,

Is this file on a network drive?  I had this similar problem because my file was on the organizations shared network drive and it was godly slow to refresh.  I said screw it and put the data on my local drive and it refreshes at least 5x faster.  

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Power BI Monthly Update - May 2024

Check out the May 2024 Power BI update to learn about new features.