Copy data activity on microsoft fabric and mongodbAtlas problem connection

[BartoloIsmail]

I'm trying to utilize the new MongoDB Atlas connector in Data Pipelines/Copy Data, but have had no luck at all connecting to my MongoDB Atlas database. It seems to be a connectivity issue or something wrong with my connect string. Can anyone help with the proper format on setting up atlas as a source? 

>Connection to MongoDB server is timeout.
A timeout occurred after 30000ms selecting a server using CompositeServerSelector{ Selectors = MongoDB.Driver.MongoClient+AreSessionsSupportedServerSelector, LatencyLimitingServerSelector{ AllowedLatencyRange = 00:00:00.0150000 }, OperationsCountServerSelector }. Client view of cluster state is { ClusterId : "1", Type : "Unknown", State : "Disconnected", Servers : [], DnsMonitorException : "DnsClient.DnsResponseException: Query 24306 => _mongodb._tcp.clustermarket.clustermarket.yvfht.mongodb.net IN SRV on 169.254.0.1:53 timed out or is a transient error. ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
at DnsClient.DnsUdpMessageHandler.Query(IPEndPoint server, DnsRequestMessage request, TimeSpan timeout)
at DnsClient.LookupClient.ResolveQuery(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit)
--- End of inner exception stack trace ---
at DnsClient.LookupClient.ResolveQuery(IReadOnlyList`1 servers, DnsQuerySettings settings, DnsMessageHandler handler, DnsRequestMessage request, LookupClientAudit audit)
at DnsClient.LookupClient.QueryInternal(DnsQuestion question, DnsQuerySettings queryOptions, IReadOnlyCollection`1 servers)
at MongoDB.Driver.Core.Misc.DnsClientWrapper.ResolveSrvRecords(String service, CancellationToken cancellationToken)
at MongoDB.Driver.Core.Clusters.DnsMonitor.Monitor()" }.

Thanks for your support

[Marusyk]

I also have the problem with connection because by connection string must contain 

&authmechanism=SCRAM-SHA-256&

how can I specify it in Fabric? There are only server name + port and creds. How to specify an additional param?

[Anonymous]

Hi @Marusyk ,

Thanks for using Fabric Community.
Can you explain me more about your issue? What are you trying to do?

[Marusyk]

Hi, sure. I'm using the RBAC feature for CosmosDb. So I've created a user fabric and want to use it to connect from Fabric.

To connect to Cosmos using custom credentials use connection string like this (replace <username> and <password>):

mongodb://<username>:<password>@cosmos****-prod.mongo.cosmos.azure.com:10255/Customers?ssl=true&replicaSet=globaldb&authmechanism=SCRAM-SHA-256&retrywrites=false&maxIdleTimeMS=120000

I got the error however the key is valid because I'm able to connect via MongoDB Compass

[Anonymous]

Hi @Marusyk ,

If you are also facing issue with connection. The best course of action is to open a support ticket and have our support team take a closer look at it.

Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link 

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

[ChrisE15890c]

I am getting the same issue and based searching the Internet this error is common and yet I have not seen one solution provided by either Microsoft or MongoDB where I have found the issue posted. The documentation for creating the connection is abysmal. No mention of how to handle authentication with only the basic option. Please at the very least get the documenation more extensive besides simply stating to create the connection. Really??

When I create the connection on my gateway it is successful. When I hit the refresh button it is successful.

When I select the connection in the pipeline copy activity and then hit next I get the results below. I have used Python code on the gateway server to confirm there are no issues with connecting to the MongoDB server such as DNS, firewall, port, etc....

Any help would be greatly appreciated but considering how old this thread is I am doubtful there will be any.

Thanks

Chris

[Anonymous]

Hi @BartoloIsmail ,

Thanks for using Fabric Community.

Here are some steps to troubleshoot the connection and set up MongoDB Atlas as a source:

1. Verify Connection String:
   • Double-check your connection string for typos or incorrect formatting.
2. Firewall Rules:
   • Ensure your network allows outbound connections to MongoDB Atlas on port 27017 (or the custom port you might be using). Check your Data Pipelines service's network configuration and any firewalls in between.

Additional Tips:

Consider increasing the timeout value in your connection string using connectTimeoutMS parameter. This might help if there's a slight delay in establishing the connection.

Docs to refer - Configure MongoDB Atlas in a copy activity - Microsoft Fabric | Microsoft Learn

Hope this helps. Do let us know incase of further queries.

[BartoloIsmail]

Is there a way to deposit a JSON file into my bronze layer of my Microsoft Fabric lakehouse in a non-manual manner using a Talend job, for example, without using the MongoDB Atlas pipeline connectors or MongoDB Atlas SQL via Gen2 flows? I've noticed that there are SQL string connections available, but they seem only usable by tools like Power BI Desktop, for instance

[Anonymous]

Hi @BartoloIsmail ,

You can use this SQL Connection String with following authentication methods.

Connectivity to data warehousing - Microsoft Fabric | Microsoft Learn

I also checked the possible connection options with Talend Cloud - List of supported connectors | Talend Cloud Data Preparation User Guide Help

You can also send your data to adls gen 2 and can pull it into lakehouse using fabric data factory.

If you like to connect with MongoDB Atlas, please follow this link - Configure MongoDB Atlas in a copy activity - Microsoft Fabric | Microsoft Learn

Hope this provided some insights. Do let me know incase of further queries.

[Anonymous]

HI @BartoloIsmail ,

We haven’t heard from you on the last response and was just checking back to see if you got some insights over your query.
Otherwise, will respond back with the more details and we will try to help .

Thanks

[BartoloIsmail]

hi @Anonymous 

I'm reaching out again because the problem persists, and I would like to get additional help for troubleshooting the connection.

[Anonymous]

Hi @BartoloIsmail ,

Apologize for the issue you are facing. The best course of action is to open a support ticket and have our support team take a closer look at it.

Please reach out to our support team so they can do a more thorough investigation on why this it is happening: Link 

After creating a Support ticket please provide the ticket number as it would help us to track for more information.

Hope this helps. Please let us know if you have any other queries.

[BartoloIsmail]

my id is TrackingID#2405160050001635

[Anonymous]

Hi @BartoloIsmail ,

Thanks for sharing the support ticket.

Please allow some time, so team can check and provide a resolution.

In case if you got a resolution please do share with the community as it can be helpful to others .

[BartoloIsmail]

Hi @Anonymous ,

this morning I tried to input 00000. IP address in order to allow all access for everyone, and now I can conclude that it's Microsoft Fabric that doesn't want to connect. I have an internal error.

I found a similar message (https://community.fabric.microsoft.com/t5/Data-Pipelines/MongoDB-Atlas-Data-Copy-Connectivity/m-p/3566910) regarding the issue I'm facing. Let me explain: I'm trying to copy data via a pipeline using Microsoft Fabric to retrieve my MongoDB Atlas collection. I had an error that I corrected by adding a parameter that allows access to all IPs to access the MongoDB service. After running the test, I now encounter a new error that appears (progress... :)). I'm coming to you to ask if you could explain where this new error might be coming from. Thank you for your help.