Azure DataBricks: Unable to Import Table with Many Rows?

[Anonymous]

Hi all,

I have data stored in Azure DataBricks that I would like to use in my Report. I have access to the DataBricks workspace and the cluster is up and running. I have also been able to import small tables from the Azure DataBricks Database (tables of ~500 rows and ~10 columns). 

However, larger tables are causing me trouble. My desired table has ~5 million rows and ~20 columns. 

Trying to import that table results in this error:

Failed to save modifications to the server. Error returned: 'OLE DB or ODBC error: [DataSource.Error] ERROR [HY000] [Microsoft][Hardy] (35) Error from server: error code: '0' error message: 'Unable to continue fetch after reconnect. Retry limit exceeded.'.. '.

Does anyone know what could be causing this? Sometimes when I try to import, I get the display of x number of rows imported so far. The number does not exceed 1,000,000 before returning this error message. 

I'd appreciate any insight to this problem. Thank you! 

[Anonymous]

Hi @Anonymous ,

Data sources for Power BI - Power BI | Microsoft Learn

Please check the limit from the above document:

• Row limit - When using DirectQuery, Power BI imposes a limit on the query results that are sent to your underlying data source. If the query sent to the data source returns more than one million rows, you see an error and the query fails. Your underlying data can still contain more than one million rows. You're unlikely to run into this limit as most reports aggregate the data into smaller sets of results.

Did you use the DirectQuery to query you source?

Best Regards,

Stephen Tao

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

[Anonymous]

Hi @Anonymous

Thank you for the response! I was not using DirectQuery to query my source. I'm actually unable to connect to my Table using DirectQuery in Azure Databricks. I've detailed that issue here: https://community.powerbi.com/t5/Desktop/Can-Import-from-Azure-DataBricks-but-cannot-Direct-Query/m-p/2806664#M971621. Haven't found a solution for it yet, so I'm trying to only use Import for now. Do you think maybe the file is too big for Power BI Desktop? It's basically all text or decimal values. Do you know if there is a way to check if PBI Desktop is unable to handle the file because of its size?