topic Re: Data Factory Lineage Issues in Dataflow

Data Factory Lineage Issues

AlanZ — Tue, 12 Dec 2023 13:28:59 GMT

Greetings,

I'm really struggling with lineage in Data Factory.

Looking at this screenshot:

First-- the Lakehouse and Warehouse on the left don't have any names/identifiers in Lineage view, even though the root objects are named clearly.

Second-- that dataflow isn't fed by a warehouse! The lineage says it is, though. Is that because it's fed by a Lakehouse-- does it create a pseudo-warehouse and show both in the lineage?

Third: When I look at the impact of that Lakehouse, it shows dataflows that absolutely aren't in the lakehouse feeding this process. It even has dataflows in a testing workspace I created.

And finally fourth: that dataflow in the original screenshot populates a warehouse. But that isn't reflected here (or anywhere I can find from a lineage perspective).

Thanks for your help!

Re: Data Factory Lineage Issues

Anonymous — Wed, 13 Dec 2023 10:51:45 GMT

Hi AlanZ
Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this.
We will update you once we hear back from them.
Thanks.

Re: Data Factory Lineage Issues

AlanZ — Wed, 13 Dec 2023 12:40:15 GMT

Thanks so much, Anonymous !

Re: Data Factory Lineage Issues

Anonymous — Thu, 14 Dec 2023 16:46:30 GMT

Hi AlanZ
Apologies for the issue.

This might require a deeper investigation from our engineering team about your workspace and the logic behind it to properly understand what might be happening.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.

Re: Data Factory Lineage Issues

AlanZ — Thu, 14 Dec 2023 19:02:18 GMT

Thanks very much, Anonymous ! Submitted, support request number 2312140040010450

Re: Data Factory Lineage Issues

Joshrodgers123 — Fri, 15 Dec 2023 20:49:57 GMT

I am seeing the same issues. The lineage views are making no sense.

I have the same nameless "Lakehouse" feeding my dataflow, which is not correct. My dataflow is feeding a lakehouse, which is seen at the bottom of the screen, not connected to the dataflow.

All of the staging artifacts are also visible. I thought those would be hidden.

I also see the same issue where completely unrelated dataflows are showing under the impact analysis of the lakehouse.