Error when refreshing on service using row limit parameter

Hi EpicTriffid 

You're dealing with two separate but related issues:

1. "Column does not exist in dataset" – This typically happens when the Keep Rows step is placed too early in the query, before Power Query finalizes the column structure (e.g., before Promoted Headers or Changed Type). Power BI Desktop might handle this leniently due to caching and more flexible evaluation, but the Power BI Service has stricter schema validation. When you change the parameter to "All" in the Service, it fails because the columns aren't guaranteed to exist at that point. To fix this, make sure your row filtering step comes after headers are promoted and types are applied.

2. "DataFormat.Error: The number of items in the list is too large" – This is a hard limit you're hitting. You're trying to load a 16GB CSV file via SharePoint – that's not a sustainable or supported approach. Power BI doesn't load this type of file in a partitioned or indexed way. Instead, it reads it row-by-row through HTTP, which easily breaks when the data volume is that high. Even in Premium capacity, the problem isn't with model size or refresh capacity – it's with the source and how it's being accessed.

What to do instead:

• Split the large CSV into multiple smaller files (e.g., by month or year), place them in a SharePoint folder, and use "Combine Files" in Power Query.

• If possible, move the data into a proper structured data source – like SQL Server, Azure Data Lake, or even Dataflows.

• Consider setting up Incremental Refresh so you're not reloading historical data on every refresh.

• Avoid using dynamic row-limit logic (like "Keep Rows" with a parameter) before schema is fully defined.

Bottom line: Power BI Service is not built to handle 15M-row CSVs over SharePoint. It may work in Desktop, but that doesn’t mean it’s production-ready. The real fix is to shift to a better data architecture – not to tweak Power Query steps to squeeze one more refresh through.

If this post helps, then please consider Accepting it as the solution and giving Kudos to help the other members find it more quickly