What is Microsoft mashup evaluation container? How it impacts the data refresh?

[Anonymous]

Hi All,

I am currently troubleshooting one data refresh issue in Power BI where we are sourcing data from Oracle Public Cloud (Import Model). When my gateway is running during refresh, I could see many instances of Microsoft mashup evaluation container running in parallel. It sucks up the RAM for which I disabled the Backgound data option in "Data load" category. I need to clarify few things here:

1. What is Microsoft mashup evaluation container?

2. Will it only run when import model is used in Power BI?

3. Is this used while sourcing the data from source only or for loading the data to Power BI xvelocity engine further?

4. I am working with 100 Mn data and applied sorting for better compression. Will Microsoft mashup evaluation container be used for this purpose also?

[GilbertQ · Accepted solution]

Hi @Anonymous

Please see my answers below.

1. What is Microsoft mashup evaluation container?

GQ - This is the Power Query Editor Engine that completes all the steps that were done in the Power Query Editor. It also compresses the data using the xVelocity Engine.

2. Will it only run when import model is used in Power BI?

GQ - Yes it will only run to import and convert all the data into the data model.

3. Is this used while sourcing the data from source only or for loading the data to Power BI xvelocity engine further?

GQ - It is also used when sourcing data, because the Power Query Editor is responsible for importing data, making changes to data and converting it into the data model.

4. I am working with 100 Mn data and applied sorting for better compression. Will Microsoft mashup evaluation container be used for this purpose also?

GQ - Yes it will still be used, it should hopefully be quicker, because it will not have to sort the data.

[Anonymous]

Hi GilbertQ,

Can you explain the fourth point in detail?

In my case one instance of Microsoft mashup evaluation container keeps running and this results into timeout.

Thanks

[GilbertQ]

Hi there,

For the fourth point, I think what is happening is because you are doing the sort operation over 100 million rows, it is taking a very long time to return the data. And this is taking longer than the timeout specified in the connection.

What you could do is to change the Timeout on the SQL Connection to be longer?
Also potentially put a clustered index on the SQL table, which should return the data immediately because when the index is created it will sort the table.

[Anonymous]

Hi GilbertQ,

Thanks for the reply. Actually in my case Power BI Desktop is not timing out. It runs successfully everytime but in Power BI Services it times out when I am refreshing a huge dataset (93Mn records). We are using P1 Premium capacity and still it times out sometime even after 2 hours.

Thanks

[GilbertQ]

Have you tried using the incremental loading?

[Anonymous]

We have requirement to have last 36 months of data to be present in the report. I tried implementing incremental refresh also but as of date, one field in the fact table should be of date time format for incremental to work. In our fact table Datekey is coming in "20180301" format. I converted this field to Date time field using Power Query but it results engine not to run native queries to the data source. This further results in bringing all data from fact table and then converting it to DateKey and apply filters.

I am thinking of creating a view in the datasource with Date Time format field now.

One question: Is there any timeout limit for on premises data gateway? It seems all refreshes which takes less than 2 hours are completing fine but beyond 2hours is timing out. Although we are using P1 capacity but still this is happening.

Thanks

[GilbertQ]

Hi there, you could create a view or put in the TSQL syntax instead of using the table from the navigation. I'm certain you could improve your query performance if you get your DBA to assist you.

I have not had data take that long to refresh. At one customer I did have 10 million rows that took 20mins to complete.