How to create dataset from dataflow?

[Anonymous]

Hi,

I have 2 dataflows, one loads the data from an SQL server and the other one applies some manipulations on the data. This data are very big, around 10 GB. I have a premium account and it is said that dataflows can handle big amount of data but I just cannot see it. I don't want to load all the data in the desktop PBI by getting data from the dataflow because I cannot upload the pbix file then (it is too big and the upload process times me out, the connecion closes and so on). What I would need (I think) is a live connection to a dataset in the workspace. Is there any way to create a dataset other then uploading from dekstop pbi? 

What would you suggest if you had data larger than 15 GB? 

Thank you in advance!

[v-robertq-msft]

Hi, @Anonymous 

According to your description, your data is stored in SQL Server and you had data larger than 15 GB. I think it’s impossible for you to import all the data into Power BI desktop because it has a data size limitation of 1GB. In my opinion, Direct Query can be the best way for you to connect with such a big size of data from SQL Server. Because it requires no large transfer of data, it's queried in place and the 1-GB dataset limitation doesn't apply to DirectQuery.

You can follow this document to connect to SQL Server using DirectQuery in Power BI Desktop:

https://docs.microsoft.com/en-us/power-bi/connect-data/desktop-use-directquery

More detail of using Direct Query in Power BI

Best Regards,

Community Support Team _Robert Qin

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.