BigQuery dataflow not refreshing

[Anonymous]

Hello, I have set up a dataflow to BigQuery using the service account associated with that google cloud project. However, automatic refresh is now disabled with the following error: 

I have refreshed the credentials several times but this has done nothing. I also tried the same procedure but with my personal organizational account but the result is the same. The data source is online so I have understood that a gateway is not necessary. Is this a problem anyone else has seen before? 

[Anonymous]

Hi @Anonymous 

From the error information , we can see that the dataflow is missing credentials . If the data source in Manage gateways is connecting successfully , do you configure it in dataflow settings ?

Best Regards,
Community Support Team _ Ailsa Tao

[Anonymous]

Hi @Anonymous 

I am not using a gateway as the data is stored in the cloud and not on-site. Have I misunderstood that gateways are not required for cloud based connections? 

When I go to the dataflow settings, the credentials they are greyed out as seen below:

When I go to the Power Platform admin center to refresh the credentials with the service account I get this message: 

I have always logged into the service account using the json key as the password and it has worked, but now it's saying the password is too long. 

[Anonymous]

Hey @Anonymous 

Any update here? I'm currently having to republish each report in my organization daily in order for my stakeholders to get updated data.

[Anonymous]

Hi @Anonymous 

This is a known issue related to the new gateway UI, please refer to this post.

Accountkey limited to 200 characters in the new interface of the Data Gateway  

Please use the link below to switch to the old manage gateway interface and try again.

(Add the following suffix ?newManageGatewaysUI=false to your original link)

https://app.powerbi.com/groups/me/gateways?newManageGatewaysUI=false 

Best Regards,
Community Support Team _ Ailsa Tao

[Anonymous]

@Anonymous As stated in my original post, I'm not using a gateway as BigQuery is a cloud based source. Why would this relate to the gateway UI?

I don't feel this has solved my problem and would like the tag turned back to 'investigating' or a more complete explanation please. 

[Anonymous]

@Anonymous I was able to coerce the json credentials into one line and edit the credentials with that. So now the dataflow refreshes but the dataset that was created from that dataflow does not. The credentials for it are greyed out and only the dataflow shows up in the power platform admin site. Since none of my dashboards will refresh, I'm still stuck. 

[Anonymous]

Hi  @Anonymous 

You can refer to the following offical document , hope it will help you .

https://docs.microsoft.com/en-us/power-query/connectors/googlebigquery#authenticating-through-a-google-service-account 

Best Regards,
Community Support Team _ Ailsa Tao

[Anonymous]

@Anonymous Sorry, I think your reply might be missing the link?

[Anonymous]

@Anonymous Yes, I coerced the json to all one line as the documentation suggests. However now the problem is not that the dataflow doesn't update (it does now), but rather that the datasets I created from the dataflow are trying to authenticate on their own and are not using the dataflow's authentication. So my datasets aren't refreshing. When I go to power platform, only the dataflow source is there: 

But when I look at the lineage view, I'm seeing a direct big query connection that completely bipasses the dataflow (I think). I'm a little confused as I published this report from desktop with the dataflow-dataset-report schema. 

Do you have any advice?