Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Refresh Error in Dataflow Connected to CosmosDB

Hello everyone,

I'm experiencing an issue while attempting to refresh my dataflow in the Power BI Service. I'm connected to Azure CosmosDB utilizing an APIKEY. Everything operates seamlessly in Power BI Desktop; I'm able to preview the data in Power Query without any hindrances. However, when attempting to refresh the data in Power BI Service, I encounter an error.

 

Error Details:

Requested on,Dataflow name,Dataflow refresh status,Table name,Partition name,Refresh status,Start time,End time,Duration,Rows processed,Bytes processed (KB),Max commit (KB),Processor Time,Wait time,Compute engine,Error,
2023-10-30 15:16:46,xxxxx,Failed,yyyyy,NA,Failed,2023-10-30 15:16:46,2023-10-30 15:16:48,00:00:01.9660,NA,NA,NA,NA,NA,NA,Error: An internal error occurred.. Param1 =  Request ID: de20fe9c-9b5a-c926-d0ec-ee41922f7b55.
 
For further clarity, I'm utilizing the following Power Query code, which was auto-generated by Power BI:
let
Source = CosmosDB.Contents("https://censured.documents.azure.com:443/"),
Navigation = Source{[Name = "censured", Kind = "Database"]}[Data],
#"Navigation 1" = Navigation{[Name = "censured", Kind = "Schema"]}[Data],
#"Navigation 2" = #"Navigation 1"{[Name = "censured", Kind = "Table"]}[Data]
in
#"Navigation 2"

 

It's worth noting that in Power BI Desktop, everything functions correctly. However, my primary need is to load data into the dataflow as this is pivotal for my ETL process.

Has anyone encountered a similar issue or have any suggestions on how to navigate this challenge?

I sincerely appreciate any insights or guidance in advance!

Dataflow
Need Help

3 Replies

  • mohitkumawat's avatar
    mohitkumawat
    Impactful Individual
    2 years ago

    Hi here are some key points to troubleshoot the dataflow refresh issue in Power BI Service:

    1. Check Data Source Permissions: Ensure your Azure Cosmos DB API key has the necessary permissions and is correctly configured.

    2. Dataflow Configuration: Verify that dataflow settings and connection details match between Power BI Desktop and Power BI Service.

    3. Dataflow Data Source: Review and confirm the data source settings within Power BI Service.

    4. Error Logging: Enable diagnostic logging in Power BI Service to gather more detailed error information.

    5. Dataflow Dependencies: Check for dependencies on other datasets or dataflows and ensure they are correctly configured.

    6. Environment and Gateway: Ensure the appropriate gateway and network configurations for connecting to Azure Cosmos DB.

    7. Data Refresh Scheduling: Check for scheduling conflicts and resource competition during refresh.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi mohitkumawat,

    Thank you for the suggestions. I've gone through each point step-by-step and here's my observation:

    Check Data Source Permissions:
    I simulated an authentication error, and this is the error I received in Power Query:
    "ODBC: ERROR [HY000] [Microsoft][DocumentDB] (401) HTTP 401 Authentication Error: [...]"
    Given the nature of this error, I believe it's not a permission issue since the error points to something else.

    Dataflow Configuration:
    The configuration is consistent across both environments. Within the Service, my only available option is Azure Cosmos DB v2.

    Dataflow Data Source:
    Upon connection, a 'Display Options' prompt allows me to select the desired database and table. There seems to be minimal room for error at this step.

    Error Logging:
    I couldn't find a way to enable diagnostic logging in Power BI Service. If you have any guides on this, I'd be keen to review. However, I did activate diagnostic logs on CosmosDB. There, I could verify that Power BI connections to Cosmos are error-free, all showing a 200 status code.

    Dataflow Dependencies:
    There aren't any. I've set up a simple dataflow without any intricate structures to ensure there are no hidden dependencies.

    Environment and Gateway:
    Connection issues between Power BI and Cosmos seem non-existent. As I mentioned earlier, from the Cosmos side, I see Power BI calls resulting in a 200 status code.

    Data Refresh Scheduling:
    This is a new workspace with premium capacity. I also enabled the "Turn on the enhanced compute engine for this dataflow", so I'm not anticipating any issues here.

    I've conducted numerous tests, with the longest refresh taking just 4 seconds.

    It's worth noting that I can view the data preview in the Power Query of the dataflow. However, when I save and trigger a refresh, I encounter the mentioned error. I'm not sure if there's a way to get a more detailed breakdown of this error, but I haven't found any solutions so far.