Forum Discussion
Using HierarchicalTranslator in Fabric copy activity
Fixed the issue partially by writing the documents as Json Files in the lakehouse and then using copy activity.
The hierarchy of the documents is maintained in mongoDB.
Super Userfabric data pipelines do not currently support HierarchicalTranslator mappings for mongodb sinks as of now only tabular mappings are supported. The reason is when you try to set the translator to HierarchicalTranslator, it silently reverts to TabularTranslator. MongoDB ends up storing keys like 'parameters.b' instead of nesting the fields. In the background fabric copy activity currently does not interpret dot notation ('parameters.b') in mappings as hierarchy when writing to MongoDB. It simply writes the key as is but not as a nested object.
With the current limitations the best workarounds that you can try is to ...
use a fabric notebook to transform and write to MongoDB using pymongo
for prod use cases requiring GUI based ETL with hierarchical outputs, consider ADF or synapse pipelines
Please 'Kudos' and 'Accept as Solution' if this answered your query.
Thanks Vinodh for the reply.
Re -
"use a fabric notebook to transform and write to MongoDB using pymongo"
This has another problem as Fabric notebook IPs are non-deterministic so i can't whitelist the mongo clusters specifically for the notebooks unless i resort to whitelisting the mongo cluster for all ip ranges, which won't be appropriate.