Showing results for 
Search instead for 
Did you mean: 

Fabric is Generally Available. Browse Fabric Presentations. Work towards your Fabric certification with the Cloud Skills Challenge.

Helper I
Helper I

Filter csv file before importing data


I have a very large csv file, which I like to import with Power Query.

However, I'd like to import only selected data from the file (not import everything and filter it afterward).

For example, I'd like to import only data, where column YEAR in the csv file has a value of 2020. 

Is it possible?

Super User
Super User

A CSV is a flat file format where there isn't a good way to filter it without scanning it, so you have to at least scan the whole file even if you don't actually load the whole file. If the file is too big for Power Query to scan in a reasonable amount of time, you'll need to pre-process it or filter it with another tool (like Python as @Waldo35 mentioned) that can scan more efficiently.

Is it still a csv file, when it is accessed through a https://... link? Or is it possible to pre-filter such connections?

It depends on the link. If it's just pointing to a file, then you've got to download it before filtering. You can only filter at the source if the source has some sort of data query engine that can do filtering.


Often data sources will provide more granular access via an API so that you can pass filters via URL. I think this might work for the resource you linked to. Clicking on the green Data API button on this page gives these suggestions:


Helper I
Helper I

Power Query usually is that prestep where u filter and transform data beforehand so that the dashboard itself is quick even tho it takes a while to load the data.


You could try loading it with Python code and using pandas to filter out the csv.

Would look something like this:


import pandas as pd
df = pd.read_csv ('file_name.csv')
df= df[df['YEAR'] == 2020]


For more info

pandas df docs 

Helpful resources

PBI November 2023 Update Carousel

Power BI Monthly Update - November 2023

Check out the November 2023 Power BI update to learn about new features.

Community News

Fabric Community News unified experience

Read the latest Fabric Community announcements, including updates on Power BI, Synapse, Data Factory and Data Activator.

Power BI Fabric Summit Carousel

The largest Power BI and Fabric virtual conference

130+ sessions, 130+ speakers, Product managers, MVPs, and experts. All about Power BI and Fabric. Attend online or watch the recordings.

Top Solution Authors