Forum Discussion
Automating Data Wrangler output
- Anonymous2 years ago
Hi Cornelia,
If I understand you correctly, you'd like to describe a dataset automatically/programatically, the way data wrangler does in python notebooks. I don't know that there is an api to use the data wrangler programatically.
what I can suggest is either using the dataframe description capabilites in spark:
https://spark.apache.org/docs/3.1.1/api/python/reference/api/pyspark.sql.DataFrame.describe.html
or generate the description directly in the dataset (assuming we're talking about pbi datasets = semantic models) using MCode
https://learn.microsoft.com/en-us/powerquery-m/table-schema
https://learn.microsoft.com/en-us/powerquery-m/table-profile
hope this helps!
Once you have the Python code generated with data wrangler you can put it in a cell of a notebook and from the notebook schedule a data pipeline run of the notebook.
Run add to pipeline in Microsoft Fabric notebooks