use Notebook to connect to Mysql workbench

[Lyndon1]

@v-nikhilan-msft  how can I use Notebook to connect to Mysql workbench

[v-nikhilan-msft · Accepted solution]

Hi @Lyndon1 

I understand that you’re trying to import a large MySQL table into your Lakehouse using Dataflow Gen2 and encountering issues due to the size of the table. Here are a few options you might consider:

1. Split the Data: If possible, you could try splitting your data into smaller chunks and then import these chunks one by one. This could potentially alleviate the load on Power Query.
   
   

2. Increase Resources: Depending on your setup, you might be able to allocate more resources to handle the large data import. This could involve increasing the memory or processing power of your system. You can purchase a higher capacity.
   https://learn.microsoft.com/en-us/fabric/enterprise/licenses
   
   

3. Destination Settings: Check the destination settings in Dataflow Gen2. Make sure you’re correctly specifying the data destination for your query and that you have the right permissions on the data source.
   Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn

Hope this helps. If the above steps do not solve your problem, then I would request you to reach out to the support team for better understanding of the issue. This might require a deeper investigation from our engineering team and they can guide you better.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.

Hope this is helpful. Please let me know incase of further queries.

[v-nikhilan-msft · Accepted solution]

Hi @Lyndon1 
So the query got resolved? Or you have any other concerns?

[Lyndon1]

@v-nikhilan-msft , how do I do this? While Notebook itself cannot connect to MySQL Workbench, you can leverage Data Factory's MySQL connector within Notebooks for data manipulation tasks.

[v-nikhilan-msft]

Hi @Lyndon1 

You can use the Mysql connector in Dataflow Gen2 to bring the data into your lakehouse. Then you can query this data with the help of Notebooks.

Hope this helps. Please let me know if you have any further questions.

[Lyndon1]

Hi @v-nikhilan-msft . I am currently trying to bring over a 11gig (65+ million rows table) to my Lakehouse, due to the sheer size, its just loads on power query and fails. For a business requirement I need the entire table.. What are my options?

[v-nikhilan-msft · Accepted solution]

Hi @Lyndon1 

I understand that you’re trying to import a large MySQL table into your Lakehouse using Dataflow Gen2 and encountering issues due to the size of the table. Here are a few options you might consider:

1. Split the Data: If possible, you could try splitting your data into smaller chunks and then import these chunks one by one. This could potentially alleviate the load on Power Query.
   
   

2. Increase Resources: Depending on your setup, you might be able to allocate more resources to handle the large data import. This could involve increasing the memory or processing power of your system. You can purchase a higher capacity.
   https://learn.microsoft.com/en-us/fabric/enterprise/licenses
   
   

3. Destination Settings: Check the destination settings in Dataflow Gen2. Make sure you’re correctly specifying the data destination for your query and that you have the right permissions on the data source.
   Dataflow Gen2 data destinations and managed settings - Microsoft Fabric | Microsoft Learn

Hope this helps. If the above steps do not solve your problem, then I would request you to reach out to the support team for better understanding of the issue. This might require a deeper investigation from our engineering team and they can guide you better.

Please go ahead and raise a support ticket to reach our support team:

https://support.fabric.microsoft.com/support
Please provide the ticket number here as we can keep an eye on it.

Hope this is helpful. Please let me know incase of further queries.

[v-nikhilan-msft]

Hi @Lyndon1 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

[Lyndon1]

Hi @v-nikhilan-msft , sorry for taking so long to get back to you. I used Gen 2 to transfer the data over in smaller chunks in 3 different tables. I then used a notebook to append the 3 tables into 1

[v-nikhilan-msft · Accepted solution]

Hi @Lyndon1 
So the query got resolved? Or you have any other concerns?

[Lyndon1 · Accepted solution]

all good for now. I will mark as a solution. But what I do need help is Fabric pricing and how it works please. its all a bit complicated to me. My trial is coming to an end and I need to purchase

[v-nikhilan-msft]

Hi @Lyndon1 
Please create a new post regarding your concern with Fabric Capacity. I will surely help.

Thanks

[v-nikhilan-msft]

Hi @Lyndon1 
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. Otherwise, will respond back with the more details and we will try to help.
Thanks

[v-nikhilan-msft]

Hi  @Lyndon1 
Thanks for using Fabric Community.
Microsoft Fabric's Notebook doesn't directly connect to MySQL Workbench. MySQL Workbench is a standalone desktop application for managing MySQL databases, while Notebook in Fabric is a service for creating and running code in notebooks.

However, Fabric offers options to connect to MySQL databases through Data Factory:

1. Dataflow Gen2: This data integration service within Fabric uses Power Query connectors. You can set up a connection using the MySQL database connector in Dataflow Gen2. Refer to Microsoft's documentation for details on setting up the MySQL database connector and using Power Query to connect: Set up your MySQL database connection - Microsoft Fabric | Microsoft Learn
   
   
2. You can also connect to SQL endpoint using other applications:
   Connectivity to data warehousing - Microsoft Fabric | Microsoft Learn
   
   

While Notebook itself cannot connect to MySQL Workbench, you can leverage Data Factory's MySQL connector within Notebooks for data manipulation tasks.

Hope this helps. Please let me know if you have any further questions.