Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
5 years ago

How to create dataset from dataflow?

Hi,

 

I have 2 dataflows, one loads the data from an SQL server and the other one applies some manipulations on the data. This data are very big, around 10 GB. I have a premium account and it is said that dataflows can handle big amount of data but I just cannot see it. I don't want to load all the data in the desktop PBI by getting data from the dataflow because I cannot upload the pbix file then (it is too big and the upload process times me out, the connecion closes and so on). What I would need (I think) is a live connection to a dataset in the workspace. Is there any way to create a dataset other then uploading from dekstop pbi? 

What would you suggest if you had data larger than 15 GB? 

 

Thank you in advance!

Need Help

1 Reply

  • v-robertq-msft's avatar
    v-robertq-msft
    Community Support
    5 years ago

    Hi, Anonymous 

    According to your description, your data is stored in SQL Server and you had data larger than 15 GB. I think it’s impossible for you to import all the data into Power BI desktop because it has a data size limitation of 1GB. In my opinion, Direct Query can be the best way for you to connect with such a big size of data from SQL Server. Because it requires no large transfer of data, it's queried in place and the 1-GB dataset limitation doesn't apply to DirectQuery.

    You can follow this document to connect to SQL Server using DirectQuery in Power BI Desktop:

    https://docs.microsoft.com/en-us/power-bi/connect-data/desktop-use-directquery

    More detail of using Direct Query in Power BI

     

    Best Regards,

    Community Support Team _Robert Qin

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.