We reached the end of the buffer - Dataflow

[AmiraBedh]

I have the following dataflow containing :

The Entity and the GL/Mapping (I am sharing only screenshots for Entity since the GL/Mapping has the same) are failing to retrieve data when I check them, the sources seems to be pointing to inexisting dataflows :

When I check the workspace it is existing but the Dataflow I get this message :

The workspace pointing to the dataflow exists but the dataflow is missing but the error message is different :

I get this error message :

We reached the end of the buffer

section Section1;

shared #"AFAS Openingsbalans" = let

  /*#"2018" = let

    Source = Sql.Database("XXXXXXXXXXX.database.windows.net", "XXXXXXXXXXX"),

    afas_vw_FinancieleMutaties_Current = Source{[Schema="afas",Item="vw_getAllEmps"]}[Data],

    #"Filtered Rows" = Table.SelectRows(afas_vw_FinancieleMutaties_Current, each ([Jaar] < 2018) and ([Administratie_code] <> 1) and ([Administratie_code] <> 2) and [Dagboek_code] = "99"),

    #"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Bedrag_saldo", "Verzamelen_op_rekeningnummer"}),

    #"Grouped rows" = Table.Group(#"Removed Other Columns", {"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Verzamelen_op_rekeningnummer"}, {{"Bedrag_Saldo", each List.Sum([Bedrag_saldo]), type number}}),

    #"Added Year" = Table.AddColumn(#"Grouped rows", "Year", each 2018),

    #"Added Period" = Table.AddColumn(#"Added Year", "Period", each 0),

    #"Renamed columns" = Table.RenameColumns(#"Added Period", {{"Bedrag_Saldo", "Amount"}})

  in

    #"Renamed columns",

  #"2019" = let

    Source = Sql.Database("XXXXXXXXXXX.database.windows.net", "XXXXXXXXXXX"),

    afas_vw_FinancieleMutaties_Current = Source{[Schema="afas",Item="vw_getAllEmps"]}[Data],

    #"Filtered Rows" = Table.SelectRows(afas_vw_FinancieleMutaties_Current, each ([Jaar] < 2019) and ([Administratie_code] <> 1) and ([Administratie_code] <> 2)),

    #"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Bedrag_saldo", "Verzamelen_op_rekeningnummer"}),

    #"Grouped rows" = Table.Group(#"Removed Other Columns", {"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Verzamelen_op_rekeningnummer"}, {{"Bedrag_Saldo", each List.Sum([Bedrag_saldo]), type number}}),

    #"Added Year" = Table.AddColumn(#"Grouped rows", "Year", each 2019),

    #"Added Period" = Table.AddColumn(#"Added Year", "Period", each 0),

    #"Renamed columns" = Table.RenameColumns(#"Added Period", {{"Bedrag_Saldo", "Amount"}})

  in

    #"Renamed columns",

  #"2020" = let

    Source = Sql.Database("XXXXXXXXXXX.database.windows.net", "XXXXXXXXXXX"),

    afas_vw_FinancieleMutaties_Current = Source{[Schema="afas",Item="vw_getAllEmps"]}[Data],

    #"Filtered Rows" = Table.SelectRows(afas_vw_FinancieleMutaties_Current, each ([Jaar] < 2020) and ([Administratie_code] <> 1) and ([Administratie_code] <> 2)),

    #"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Bedrag_saldo", "Verzamelen_op_rekeningnummer"}),

    #"Grouped rows" = Table.Group(#"Removed Other Columns", {"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Verzamelen_op_rekeningnummer"}, {{"Bedrag_Saldo", each List.Sum([Bedrag_saldo]), type number}}),

    #"Added Year" = Table.AddColumn(#"Grouped rows", "Year", each 2020),

    #"Added Period" = Table.AddColumn(#"Added Year", "Period", each 0),

    #"Renamed columns" = Table.RenameColumns(#"Added Period", {{"Bedrag_Saldo", "Amount"}})

  in

    #"Renamed columns",

  #"2021" = let

    Source = Sql.Database("XXXXXXXXXXX.database.windows.net", "XXXXXXXXXXX"),

    afas_vw_FinancieleMutaties_Current = Source{[Schema="afas",Item="vw_getAllEmps"]}[Data],

    #"Filtered Rows" = Table.SelectRows(afas_vw_FinancieleMutaties_Current, each ([Jaar] < 2021) and ([Administratie_code] <> 1) and ([Administratie_code] <> 2)),

    #"Removed Other Columns" = Table.SelectColumns(#"Filtered Rows",{"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Bedrag_saldo", "Verzamelen_op_rekeningnummer"}),

    #"Grouped rows" = Table.Group(#"Removed Other Columns", {"Administratie_code", "Rekeningnummer", "Rekening_omschrijving", "Verzamelen_op_rekeningnummer"}, {{"Bedrag_Saldo", each List.Sum([Bedrag_saldo]), type number}}),

    #"Added Year" = Table.AddColumn(#"Grouped rows", "Year", each 2021),

    #"Added Period" = Table.AddColumn(#"Added Year", "Period", each 0),

    #"Renamed columns" = Table.RenameColumns(#"Added Period", {{"Bedrag_Saldo", "Amount"}})

  in

    #"Renamed columns", */

[AmiraBedh · Accepted solution]

I ended up hardcoding the steps for 2 tables imported from a dataflow.

I will open a ticket to Microsoft to check why the Dataflows ID where changed since we didn't perform any actions from our side.

[v-yilong-msft]

Hi @AmiraBedh ,

The error message “We reached the end of the buffer” suggests there might be an issue with the data retrieval process. This could be due to a timeout or a problem with the data source itself. So my suggestion would be to reconnect the dataflow or check the data source to make sure the dataflow is pointing to the correct and accessible data source. Double check the connection string and credentials.

Another thing, since the workspace exists but lacks dataflow, I think you can try to restore the dataflow from a backup, which will open the dataflow settings in the Admin portal. You can look at this document: Troubleshoot dataflow issues - creating dataflows - Power Query | Microsoft Learn

Best Regards

Yilong Zhou

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.