Power BI is turning 10! Tune in for a special live episode on July 24 with behind-the-scenes stories, product evolution highlights, and a sneak peek at what’s in store for the future.
Save the dateEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Hi,
I'm wondering if it's possible to write sql query to filter data with dataflow gen2 when using a table from a lakehouse in fabric?
Any good ideas on how to solve this?
Thanks!
Solved! Go to Solution.
Hi @pbi_taken ,
You can write sql queries to filter the tables in lakehouse.
First you need to create a Dataflow Gen2 then select lakehouse as the destination, publish it and find that lakehouse then create a new notebook and write sql query to filter the tables. If you want to save the filtered results you need to use python query.
%%sql
SELECT *
FROM lakehouse_table
WHERE column_name = 'desired_value'
%%pyspark
# Read the original table
df = spark.read.table("original_table_name")
# Apply the filter
filtered_df = df.filter(df["column_name"] == "desired_value")
# Save the filtered data as a new table
filtered_df.write.mode("overwrite").saveAsTable("original_table_name")
Best Regards,
Neeko Tang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @pbi_taken ,
You can write sql queries to filter the tables in lakehouse.
First you need to create a Dataflow Gen2 then select lakehouse as the destination, publish it and find that lakehouse then create a new notebook and write sql query to filter the tables. If you want to save the filtered results you need to use python query.
%%sql
SELECT *
FROM lakehouse_table
WHERE column_name = 'desired_value'
%%pyspark
# Read the original table
df = spark.read.table("original_table_name")
# Apply the filter
filtered_df = df.filter(df["column_name"] == "desired_value")
# Save the filtered data as a new table
filtered_df.write.mode("overwrite").saveAsTable("original_table_name")
Best Regards,
Neeko Tang
If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.
Hi @pbi_taken
Yes, you can filter data in a Dataflow Gen2 using SQL when working with a Lakehouse table in Fabric. Here's how you can approach it:
Use SQL Queries in Dataflow Gen2:
Create a Dataflow:
SQL Query Transformation:
Custom SQL Filter:
SELECT *
FROM lakehouse_table
WHERE [column_name] = 'desired_value'
Pushdown Filters:
Did I answer your question? Mark my post as a solution, this will help others!
If my response(s) assisted you in any way, don't forget to drop me a "Kudos" 🙂
Kind Regards,
Poojara
Data Analyst | MSBI Developer | Power BI Consultant
Please Subscribe my YouTube for Beginners/Advance Concepts: https://youtube.com/@biconcepts?si=04iw9SYI2HN80HKS
User | Count |
---|---|
73 | |
70 | |
38 | |
24 | |
23 |
User | Count |
---|---|
96 | |
93 | |
50 | |
41 | |
40 |