CansuT's avatar
CansuT
Regular Visitor
1 year ago
Status:
Investigating

Incremental Refresh giving Expression.Error: There weren't enough elements in the enumeration to com

Hi,

I have 3 folders that I am importing from sharepoint into power query. Each of these folders include multiple csv files from different periods, so either a file per year or quarter.

I am consolidating them and applying incremental refresh as explained here . and in this blogpost.

I tested the refresh of this dashboard on power bi service after publishing, before I setup the incremental refresh on power bi desktop (so before I did the step that starts at minute 8:30 in the video I linked above) and the refresh was successfull (and limited to the files that are filtered using the rangeStart and rangeEnd in power query, as expected). 

 

But after I set up the incremental refresh as shown in the video, I start getting this error:

Expression.Error: There weren't enough elements in the enumeration to complete the operation.. #table({"Content", "Name.1", "Extension", "Date accessed", "Date modified", "Date created", "Attributes", "Folder Path", "Name", "Date"}, {}). ;There weren't enough elements in the enumeration to complete the operation.. The exception was raised by the IDbCommand interface.

 

My first guess is that one of the tables returns no rows after being filtered based on the parameters for today's incremental refresh dates (that would be the last 3 months, including today). I went in and checked the filtering steps on the tables, and when using the rangeStart and rangeEnd that would match today's ranges, all these tables return the rows as expected, none of them are empty. So this doesn't explain the issue.

 

Second mystery is that none of the tables have the columns described in the error: the columns in the error look similar to the columns one gets when connecting to a sharepoint folder, however none of the steps in tables have the Name.1 column. So I am not sure which of the tables are causing the issue. 

 

Here is the code for one of the tables, the link removed for security reasons:


 

let
    Source = SharePoint.Files(...),
    #"Filtered Rows" = Table.SelectRows(Source, each ([Folder Path] = "...")),
    #"Create DateTime for Incremental" = Table.AddColumn(#"Filtered Rows", "Text Range", each Text.Middle([Name], 20, 7), type text),
    #"Replaced Value" = Table.ReplaceValue(#"Create DateTime for Incremental","_"," ",Replacer.ReplaceText,{"Text Range"}),
    #"Changed Type3" = Table.TransformColumnTypes(#"Replaced Value",{{"Text Range", type date}}),
    #"Calculated End of Month" = Table.TransformColumns(#"Changed Type3",{{"Text Range", Date.EndOfMonth, type date}}),
    #"Changed Type" = Table.TransformColumnTypes(#"Calculated End of Month",{{"Text Range", type datetime}}),
    #"Applied Incremental Refresh Filter" = Table.SelectRows(#"Changed Type", each [Text Range] >= RangeStart and [Text Range] < RangeEnd),
    #"Invoke Custom Function1" = Table.AddColumn(#"Applied Incremental Refresh Filter", "Transform File (2)", each #"Transform File (2)"([Content])),
    #"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
    #"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform File (2)"}),
    #"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform File (2)", Table.ColumnNames(#"Transform File (2)"(#"Sample File (2)"))),
    // Replace "n/a" with an empty string
    #"Replaced Value2" = Table.ReplaceValue(#"Expanded Table Column1", "n/a", "", Replacer.ReplaceText, {"Share To All (%)"}),

    // Change date type with locale to correctly interpret European format
    #"Changed Type with Locale" = Table.TransformColumnTypes(#"Replaced Value2", {{"Date", type date}}, "nb-NO"),

    // Consolidate all other type changes
    #"Changed Types" = Table.TransformColumnTypes(#"Changed Type with Locale", {...
    }),

    // Split Hour column by delimiter and rename immediately
    #"Split Hour Column" = Table.SplitColumn(#"Changed Types", "Hour", Splitter.SplitTextByEachDelimiter({":"}, QuoteStyle.Csv, false), {"Hour", "Minute"}),

    // Remove unnecessary "Minute" column
    #"Removed Columns" = Table.RemoveColumns(#"Split Hour Column", {"Minute"}),

    // Rename and translate columns
    #"Renamed Columns" = Table.RenameColumns(#"Removed Columns", {
        ...
    }),

    // Replace "Males" and "Females" with Norwegian equivalents
    #"Replaced Demographics" = Table.ReplaceValue(#"Renamed Columns", each [Demographic], each if [Demographic] = "Males" then "menn" else if [Demographic] = "Females" then "kvinner" else [Demographic], Replacer.ReplaceValue, {"Demographic"}),

    // Add Demographic Ranking column
    #"Added Demographic Ranking" = Table.AddColumn(#"Replaced Demographics", "Demographic Ranking", each 
        if [Demographic] = "10 år+" then 1
        else if [Demographic] = "10 - 19 år" then 2
        else if [Demographic] = "20 - 29 år" then 3
        else if [Demographic] = "30 - 49 år" then 4
        else if [Demographic] = "50 - 66 år" then 5
        else if [Demographic] = "67 år+" then 6
        else if [Demographic] = "18 - 29 år" then 7
        else if [Demographic] = "19 - 24 år" then 8
        else if [Demographic] = "25 - 29 år" then 9
        else if [Demographic] = "30 - 39 år" then 10
        else if [Demographic] = "40 - 49 år" then 11
        else if [Demographic] = "40 - 55 år" then 12
        else if [Demographic] = "10 -49 år" then 13
        else if [Demographic] = "62 - 78 år" then 14
        else if [Demographic] = "kvinner" then 15
        else if [Demographic] = "menn" then 16
        else 20),

    // Final type changes
    #"Final Type Changes" = Table.TransformColumnTypes(#"Added Demographic Ranking", {
        {"Hour", Int64.Type}, 
        {"Demographic Ranking", Int64.Type}, 
        {"Lyttetid blant lytterne (minutter)", Int64.Type}
    })
in
    #"Final Type Changes"

 

 

2 Comments

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi  CansuT ,

     

    Please make sure that the indentation in your Power Query M code is correct. Improper indentation can cause unexpected errors. and double-check the filtering steps that use RangeStart and RangeEnd to ensure they are correctly filtering the data. The columns used in the filter exist and must have the correct data types.

    None of the tables return empty results after applying the incremental refresh filter. Even if the tables return rows when you manually check the filter, there might be cases where the filter results in an empty table during the actual refresh.

    If it does not help, try add debugging steps to your code to print out in termediate results and column names. This can help you pinpoint where the issue is occurring.

     

    Best regards.
    Community Support Team_Caitlyn

     

     

  • CansuT's avatar
    CansuT
    Regular Visitor

    "If it does not help, try add debugging steps to your code to print out in termediate results and column names. This can help you pinpoint where the issue is occurring."
    How can I add debugging steps to print out intermediate results and column names?