Datamarts and the "linked entity" concept

[ScottPowellECU]

Hi,

with dataflows, I can have an "ingest" dataflow that brings raw data in and persists it in the enhanced compute engine. I can then have dependent dataflows that use linked entities to perform additional transformations at database speeds.

Is there a similar capability with datamarts? I'm trying to pull in a lot of audit log data from text files and then apply additional transformations to the log data. But if I try to "reference" the raw data in a datamart, I'm getting an error that query folding is not allowed and the job blows up.

This is pretty frustrating as I know the data is getting persisted into Azure SQL server...but I can't figure out how to load the raw data first, and then have lots of other tables generated from that.

Thanks in advance!

Scott

[v-jianboli-msft · Accepted solution]

Hi @ScottPowellECU ,

Please take "incorporate both dataflows and datamarts" into consideration.

In some cases it can be useful to incorporate both dataflows and datamarts in the same solution. The following situations could find incorporating both dataflows and datamarts advantageous:

• For solutions with existing dataflows:

  • Easily consume the data with datamarts to apply any additional transformations or enable ad-hoc analysis and querying using SQL queries
  • Easily integrate a no-code data warehousing solution with no management of datasets

• For solutions with existing datamarts:

  • Perform reusable extract, transform and load (ETL) at scale for large data volumes
  • Bring your own data lake and use dataflows as a pipeline for datamarts

For more details, please refer to:

Introduction to datamarts - Power BI | Microsoft Docs

Understand datamarts (preview) - Power BI | Microsoft Docs

Best Regards,

Jianbo Li

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.