topic Re: Retrieve column descriptions from semantic models in Data Science

Retrieve column descriptions from semantic models

Anonymous — Mon, 25 Nov 2024 17:32:30 GMT

I'm trying to write a kind of "data glossary" report, where I went to detail all of the column and column descriptions from various reports. I'm trying to use the SemPy library to do this, but can't seem to figure out how to get the column descriptions.

I've looked at several articles, e.g., this one: (25) Document your fabric datasets using the semantic link and ChatGPT API | LinkedIn and can successfully get a list of all tables and their columns, but I can't get their column descriptions.

# import the module

import sempy.fabric as fabric

# set the semantic model

sm = "TableName"

# add table info to dataframe

df = spark.createDataFrame(fabric.list_tables(sm, include_columns=True, include_partitions=False, extended=False, workspace="WorkspaceName"))

display(df)

I believe using the FabricDataFrame Class (sempy.fabric.FabricDataFrame class | Microsoft Learn) might be the way forward, but can't figure out how to use it.

The plan going forward is to then iterate through several workspaces and datasets, and store all column descriptions in one place, but right now, I can't access the column descriptions

Re: Retrieve column descriptions from semantic models

Anonymous — Tue, 26 Nov 2024 02:57:19 GMT

Hi Anonymous,

I'd like to suggest you try to use Semantic functions to list all of the workspace items: (it will include the id, name, description, type and workspace id)

import sempy.fabric as fabric # Get the items of this workspace workspace_items=fabric.list_items() # Filter by type to get DataPipeline list #itemList= workspace_items[workspace_items.Type == "DataPipeline"] display(workspace_items)

Semantic functions - Microsoft Fabric | Microsoft Learn

Regards,

Xiaoxin Sheng

Re: Retrieve column descriptions from semantic models

Anonymous — Tue, 26 Nov 2024 09:21:47 GMT

Thanks OR the response, but that hasn't answered my question. I already have a list of workspaces and semantic models. I can also get the columns that are in the semantic models, but what I'm struggling tog et is the column d3escriptions. Do you know how I can get these?

Re: Retrieve column descriptions from semantic models

frithjof_v — Tue, 26 Nov 2024 20:09:26 GMT

This works for me:

import sempy.fabric as fabric display(fabric.list_columns("DatasetName"))

Well, I don't actually have column descriptions in my test dataset... But if I had, I think they would show here: