Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Earn a 50% discount on the DP-600 certification exam by completing the Fabric 30 Days to Learn It challenge.

Reply
blesd
Frequent Visitor

Large csv data file of about 30gb

I have created a Power BI report using a sub-set of a large dataset. I would like to update the data source with a csv data file of about 30gb

But it's taking too long and the remote desktop keeps breaking off, please is there a walk around this.

I would really appreciate your response, the solution is required urgently.

 

1 ACCEPTED SOLUTION
blesd
Frequent Visitor

Thanks.

These are the steps I took to make it work
1. I changed the following data load options settings:
     Unchecked auto date/time
     Disabled parallel loading of tables
     Unchecked the background data option 

2. I also increased the data management cache option and used a system with a higher ram.

View solution in original post

3 REPLIES 3
blesd
Frequent Visitor

Thanks.

These are the steps I took to make it work
1. I changed the following data load options settings:
     Unchecked auto date/time
     Disabled parallel loading of tables
     Unchecked the background data option 

2. I also increased the data management cache option and used a system with a higher ram.

v-yueyunzh-msft
Community Support
Community Support

Hi , @blesd 

 If you are a Power BI free or pro user, when you publish a PBIX file over 1GB to Power BI Service, you will get a file size limit message, but there is no limit about file size in Power BI Desktop, you should be able to load data from CSV files to Power BI Desktop.

In addition, you can load data from CSV files to SQL Server table, and you can use DirectQuery mode to connect to the SQL Server source in Power BI Desktop when you have large amount data in the table.

For more information, you can refer to :

Import csv into SQL server (with query OR without query using SSMS) - QA With Experts

Using DirectQuery in Power BI - Power BI | Microsoft Learn

 

Best Regards,

Aniya Zhang

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly

aj1973
Community Champion
Community Champion

Hi @blesd 

With a pro license you can only have a dataset up to 1G of size. Not sure what could the impact of a your CSV file on the size of the Dataset when you import it...if it is more than 1G then the desktop will time out.

Best solution would be to use the Direct Query mode connection to your CSV file.

Regards
Amine Jerbi

If I answered your question, please mark this thread as accepted
and you can follow me on
My Website, LinkedIn and Facebook

Helpful resources

Announcements
RTI Forums Carousel3

New forum boards available in Real-Time Intelligence.

Ask questions in Eventhouse and KQL, Eventstream, and Reflex.

MayPowerBICarousel

Fabric Monthly Update - May 2024

Check out the May 2024 Fabric update to learn about new features.

LearnSurvey

Fabric certifications survey

Certification feedback opportunity for the community.