Skip to main content
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Helper I
Helper I

Filter csv file before importing data


I have a very large csv file, which I like to import with Power Query.

However, I'd like to import only selected data from the file (not import everything and filter it afterward).

For example, I'd like to import only data, where column YEAR in the csv file has a value of 2020. 

Is it possible?

Super User
Super User

A CSV is a flat file format where there isn't a good way to filter it without scanning it, so you have to at least scan the whole file even if you don't actually load the whole file. If the file is too big for Power Query to scan in a reasonable amount of time, you'll need to pre-process it or filter it with another tool (like Python as @Waldo35 mentioned) that can scan more efficiently.

Is it still a csv file, when it is accessed through a https://... link? Or is it possible to pre-filter such connections?

It depends on the link. If it's just pointing to a file, then you've got to download it before filtering. You can only filter at the source if the source has some sort of data query engine that can do filtering.


Often data sources will provide more granular access via an API so that you can pass filters via URL. I think this might work for the resource you linked to. Clicking on the green Data API button on this page gives these suggestions:


Helper I
Helper I

Power Query usually is that prestep where u filter and transform data beforehand so that the dashboard itself is quick even tho it takes a while to load the data.


You could try loading it with Python code and using pandas to filter out the csv.

Would look something like this:


import pandas as pd
df = pd.read_csv ('file_name.csv')
df= df[df['YEAR'] == 2020]


For more info

pandas df docs 

Helpful resources

Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City


Power BI Monthly Update - April 2024

Check out the April 2024 Power BI update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Solution Authors
Top Kudoed Authors