Spark Connector Scheduled Refresh Failing

[chiburi]

I've been POC'ing loading data from Spark into power BI.

All was going fine until I started load testing the scheduled refresh in the power service and I'm hitting lots of different intermittant issues. I've had various errors and can't seem to establish 1) a pattern 2) a cause and 3) a solution.

I've tried upping the cluster size and memory to no avail. Anyone got any ideas?

Errors that have occured intermintantly i.e. sometimes it succeeds:

• ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Could not refill buffer. Table: ******* 
• ODBC: ERROR [HY000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'. Table: *********
• The table has no visible columns and cannot be queried. Table: *****

I get this sometimes also...

• Last refresh succeeded: null

The connection driver seems really flakey!

[Anonymous]

HI @chiburi,

Can you please share more detailed error message and some trace log to help us clarify your issue?

Fiddler - Free Web Debugging Proxy - Telerik

Regards,

Xiaoxin Sheng

[chiburi]

Hi.

I know what fiddler is but how are you expecting me to get further detail using it?

This is all Cloud Azure - powerbi app service is reshreshing against databricks spark cluster PaaS service.

Thanks

[Anonymous]

HI @chiburi,

>>I know what fiddler is but how are you expecting me to get further detail using it?

Please trace some logs for refresh operation.

Actually, I want to confirm if your issue is same as one support ticket which other user submitted.

BTW, azure has canceled spark subscription, I guess your issue may related to it.

Regards,

Xiaoxin Sheng

[chiburi]

Hi.

Which other support ticket?

"BTW, azure has canceled spark subscription, I guess your issue may related to it."

Sorry this makes no sense to me. Spark is available in Azure with Data Bricks or HDInsight. What specifically do you mean?

Thanks

[Anonymous]

Hi @chiburi,

The ticket is opened on CRI 81831666 about refresh issue with spark and odbc driver. I'm not sure if your issue is same as his.

Regards,

Xiaoxin Sheng

[chiburi]

Hi.

I'm not sure either. I googled the reference... How can I see the details of the issue?

Do you have a link?

Thanks

[sumitg]

I am also facing the same issue when i am importing data using spark connector.

     Error: ODBC: ERROR [HY0000] [Microsoft][Hardy] (34) Error from server: Invalid method name: 'GetPrimaryKeys'

Please help me with this if you have any resolution of this error

[chiburi]

Hi.

We ended raising a support call. It's an issue with the simba driver that they're using. They're working on a fix... I don't know if it will be rolled out across all tenacies in the same time frame but fix would be rolled out to everyone at some point I imagine.