Load large Mongo DB table into lakehouse, it shows timeout

[Charlotte_chum]

Hi everyone!

I am ingesting a table from mongo DB (size 1.5gb), and I am using F2 SKU.

It has been loading for 48 hours and still showing timeout. 

Just wondering is there any way to ingest the data? 

[nilendraFabric · Accepted solution]

Hello @Charlotte_chum 

The MongoDB Data Factory Beta connector provides better throughput than legacy methods, not sure which one you're using 

Use the MongoDB Atlas Connector in Data Factory pipelines for direct ingestion

On the other hand The F2 SKU has limited compute capacity, which can throttle parallelism and cause queuing.

Upgrade to F4/F8 if possible. 

If these steps fail, consider exporting MongoDB data to Azure Blob Storage first and using short-term external staging to bypass connector limitations

https://www.mongodb.com/developer/products/atlas/unlock-value-data-mongodb-atlas-intelligent-analyti...