Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Dataset fail to refresh regularly: The XXX column does not exist in the rowset

We have several PBI report with parquet files stored in ADLSG2 as datasource.  There are multiple parquet files per dataset. To load all the parquet files at once we added below line in advance editor:

 

  Source = AzureStorage.DataLake("https://xxxxxxxxxx.dfs.core.windows.net/Container"),

 

The reports are working fine on desktop. They are working after published to PBI service. The dataset can refresh successfully. So we setup schedule to refresh them 8 times per day. There are new data written to parquet everyday but no change on schema.

 

However,  roughly about once per week. The dataset fail to refresh. The error is always:

The XXX column does not exist in the rowset

 

It can be fixed by simply refresh the report data with Power BI Desktop, then publish the report again. Then the dataset will be resumed to normal for a few days until it fail again with same error.

 

Any idea what is the cause?

Need Help

7 Replies

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      According to my test, the parquet connector can only read a specific parquet file. But we need it read a specific folder with several parquet partitions and files.

      • GilbertQ's avatar
        GilbertQ
        Super User
        2 years ago

        Hi Anonymous 

         

        In Power BI it is possible to read multiple files in a folder and then connect to the Parquet files.

         

         

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi Anonymous ,
    Based on the information you have provided, the problem you are experiencing may be caused by a change in the original data source, such as a column being moved, renamed, or deleted.
    Please check if the original data source has changed.

     

    Best Regards,

    Ada Wang

    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.