Forum Discussion
Solved
Store Activity Data into a Table
I am wanting to store the data collected from the Get Metadata activity into a lakehouse table, along with the Copy Data activity output for each table loaded in the ForEach activity. I have tried us...
The following works;
1) Feed the output of the Get MetaData activity into a parameter of a Notebook Activity@string(activity('Get Metadata1').output.childItems)
2) Have a Notebook similiar to the following - attach it to a lakehousemetadata_json = json.loads(metadata)
df = spark.createDataFrame(metadata_json)
df.write.mode('overwrite').saveAsTable('get_metadata')3) When the pipeline is run, this is what gets output;
If this helps, please consider Accepting as a Solution to help others find it more easily.
Thanks! I will give that a try!
The following works;
1) Feed the output of the Get MetaData activity into a parameter of a Notebook Activity
@string(activity('Get Metadata1').output.childItems)
2) Have a Notebook similiar to the following - attach it to a lakehouse
metadata_json = json.loads(metadata)
df = spark.createDataFrame(metadata_json)
df.write.mode('overwrite').saveAsTable('get_metadata')
df = spark.createDataFrame(metadata_json)
df.write.mode('overwrite').saveAsTable('get_metadata')
3) When the pipeline is run, this is what gets output;
If this helps, please consider Accepting as a Solution to help others find it more easily.