Handling Large datasets in power bi

[Shabb]

Hi all, 

My first time here 😊

Till yesterday I worked on power Bi on my Pc only, without the service. 

Today at my work we are trying to use power bi service Pro to get data from database and present a live data for a specific department.

As of my understanding the first step has to be always build and load data first on power bi desktop and then publish to bi service (which will create live connection directly to the database and refresh data) ,  am I mistaken? Can I use loading data faster on bi service? 

My problem is that the data model is relatively big and loading all table rows on power bi desktop(which is after all made on a small Pc) before publishing takes forever 

First question :

Do I have to load all data into Power bi desktop before publishing? Is there a way to load samples only and after publishing it will load full data from the bi service engine which should be faster? 

Second question :

Does the bi service refreshes directly from the database?  If it does, then my first question is still relevant.

I am asking this because eventually we need live data on the bi service dashboard (max 30 minutes refresh cardinality)  is that even possible?

Thanks in advance,

[Anonymous · Accepted solution]

Hi @Shabb ,

First question :

Q: Do I have to load all data into Power bi desktop before publishing?

A: 1.You can also create report by power bi service, but in this way, you cannot create measure and  model

    2. So far, Power BI provide three connection types for connection with data source

         Import: will import all data into model

         Direct Query: does not import all data into model, will query data according to your demands
         Live Connect: same as Direct Query

Q: Is there a way to load samples only and after publishing it will load full data from the bi service engine which should be faster? 

A: No

Second question :

Q: Does the bi service refreshes directly from the database? 

A: Yes

Q: I am asking this because eventually we need live data on the bi service dashboard (max 30 minutes refresh cardinality)  is that even possible?

A: You can choose Qirect Query or Live Connect connection model

For more details, you can read related document: Using DirectQuery in Power BI - Power BI | Microsoft Docs ,  Power BI Connection Types -

Best regards,
Community Support Team_ Binbin Yu
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anand24]

Hi @Shabb ,

PFB inline answers:

First question :

Do I have to load all data into Power bi desktop before publishing? Yes

Is there a way to load samples only and after publishing it will load full data from the bi service engine which should be faster? No

Second question :

Does the bi service refreshes directly from the database? Yes

If it does, then my first question is still relevant. It refreshes directly after 1st load in desktop is you are going by the correct method of creating report in Desktop and then publishing to Services

I am asking this because eventually we need live data on the bi service dashboard (max 30 minutes refresh cardinality)  is that even possible? For live data, connect your data sources in DirectQuery storage mode

Give a Thumbs Up if this post helped you in any way and Mark This Post as Solution if it solved your query !!! Proud To Be a Super User !!! LinkedIn