Forum Discussion

DarrenCooper's avatar
DarrenCooper
Frequent Visitor
9 months ago
Solved

Issue With Google BiqQuery Connection in Dataflow 2. 'LargeResultDataset' are not valid BigQuery

Hi, I'm trying to test ingesting data from Google big query via Datflow Gen 2. I keep getting the below error at the connection stage , how do I get around it? An exception occurred: 'LargeR...
Need Help
  • v-dineshya's avatar
    v-dineshya
    9 months ago

    Hi DarrenCooper ,

    Thank you for reaching out to the Microsoft Community Forum.

     

    The error 'LargeResultDataset' are not valid BigQuery options is due to the Dataflow Gen 2 service passing an unsupported parameter to BigQuery during connection setup.


    The 'LargeResultDataset' parameter is not a valid option for BigQuery connections in Dataflow Gen 2. Please refer below the valid options currently supported.

     

    1. BillingProject
    2. Implementation
    3. UseStorageApi
    4. ConnectionTimeout
    5. CommandTimeout
    6. ProjectId

     

    Note: Even if you haven't explicitly set LargeResultDataset, it seems the system might be injecting it automatically due to a bug or misconfiguration in the underlying Power Query M code.

     

    Please try below steps.

     

    1. Open the Advanced Editor in your Dataflow setup. Check the BigQuery connection line, like below syntax.

     

    GoogleBigQuery.Database([LargeResultDataset = null, ...])


    2. Replace above syntax with below syntax.

     

    GoogleBigQuery.Database([
    UseStorageApi = null,
    ConnectionTimeout = null,
    CommandTimeout = null,
    BillingProject = "billing-project-id"
    ])


    3. Save and revalidate the connection.

     

     

    I hope this information helps. Please do let us know if you have any further queries.

     

    Regards,

    Dinesh

v-dineshya's avatar
v-dineshya
Community Support
9 months ago

Hi DarrenCooper ,

Thank you for reaching out to the Microsoft Community Forum.

 

The error 'LargeResultDataset' are not valid BigQuery options is due to the Dataflow Gen 2 service passing an unsupported parameter to BigQuery during connection setup.


The 'LargeResultDataset' parameter is not a valid option for BigQuery connections in Dataflow Gen 2. Please refer below the valid options currently supported.

 

1. BillingProject
2. Implementation
3. UseStorageApi
4. ConnectionTimeout
5. CommandTimeout
6. ProjectId

 

Note: Even if you haven't explicitly set LargeResultDataset, it seems the system might be injecting it automatically due to a bug or misconfiguration in the underlying Power Query M code.

 

Please try below steps.

 

1. Open the Advanced Editor in your Dataflow setup. Check the BigQuery connection line, like below syntax.

 

GoogleBigQuery.Database([LargeResultDataset = null, ...])


2. Replace above syntax with below syntax.

 

GoogleBigQuery.Database([
UseStorageApi = null,
ConnectionTimeout = null,
CommandTimeout = null,
BillingProject = "billing-project-id"
])


3. Save and revalidate the connection.

 

 

I hope this information helps. Please do let us know if you have any further queries.

 

Regards,

Dinesh

  • v-dineshya's avatar
    v-dineshya
    Community Support
    9 months ago

    Hi DarrenCooper .

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

     

    Regards,

    Dinesh

    • v-dineshya's avatar
      v-dineshya
      Community Support
      9 months ago

      Hi @DarrenCooper .

      We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.

       

      Regards,

      Dinesh