Forum Discussion
Advocate IITable level or Columns level lineage
Hallo Community,
is it possible to trace the source tables and columns used in a report?
I understand that intermediate transformations can change the tables and columns during the process. However, I would like to know if there is any workaround or best practice to trace the fields in a report back to their original source tables and columns.
We are having Medallion architecture and using Noteboos (pyspark) to transform the data.
Any tips or suggestions would be greatly appreciated.
Hallo,
Yes, it is possible — and the fabric-toolbox repository provides a ready-made solution for this exact scenario.
Microsoft's fabric-toolbox includes a Lineage Extractor notebook that automatically extracts column-level metadata from Fabric artifacts and uploads it to Microsoft Purview to create a graphical lineage graph.
You can take a look at
https://github.com/microsoft/fabric-toolbox/tree/9dcbc14496f4f6342f3af1f482073ff0be0d91e0/tools/Lineage_Extractor
7 Replies
EduardoCastroHelper I
Hallo,
Yes, it is possible — and the fabric-toolbox repository provides a ready-made solution for this exact scenario.
Microsoft's fabric-toolbox includes a Lineage Extractor notebook that automatically extracts column-level metadata from Fabric artifacts and uploads it to Microsoft Purview to create a graphical lineage graph.
You can take a look at
https://github.com/microsoft/fabric-toolbox/tree/9dcbc14496f4f6342f3af1f482073ff0be0d91e0/tools/Lineage_Extractor
KuladeepI think this works solution upto some extent of our usecase. I will test this and get back here
- Kuladeep
I am already aware of it. and also the item level lineage.
Interested to know if anyone in the community has developed any work around or tricks to achieve table level lineage. Or at least list all the related tables (bronze till gold) to the report 
Murtaza_GhafoorSuper User
Hi Kuladeep,
Recommended: For table level lineage use Fabric Lineage view.Fabric can provide lineage across items such as:
Source → Bronze → Silver → Gold → Semantic Model → Report
This is the easiest way to understand dependencies between tables, notebooks, semantic models, and reports.
If you need column level lineage then you need to build meta data table across your workflow, this is where things will become difficult.
you can create this by using DAX studio butHowever, DAX Studio will not give you complete lineage back through your PySpark notebooks, it will only limit up to sematic layer, for lineage back to pyspark you must create techinal meta data table in your workspace using pyspark. Something like this
Target LayerTarget TableTarget Column Source TableSource
If this helps, ✓ Mark as Kudos | Help Other
v-kathullacCommunity Support
Hi Kuladeep
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
Regards,
Chaithanya- v-kathullac
Hi Kuladeep
As we haven’t heard back from you, we wanted to kindly follow up to check if the solution provided for the issue worked? or Let us know if you need any further assistance?
Regards,
Chaithanya
