Forum Discussion
Moving from Power BI to data engineering
Hi,
I learnt a bit of SQL + entity relationship model during the university, but I haven't had much working experience on it.
I have a basical knowledge of Github and code versioning, that's why I don't like low-code (no control versioning).
I also recently took Microsoft Fabric DP-600 certification: it's something that I would like to work on, and it has both low code and pro-code part.
My next step will be build a little ETL pipeline.
Thank you for your advice
Hi,
thank you for tour advices!
This post can be closed
Hello BIA1996
You are in a good position already, as you have been working with DAX and M language. Your direction of travel to learn a code based approach also seems appropriate. If you've already done DP-600 your best bet is to try Notebook approach using SQL or Python language, and get some experience in working with Spark APIs.
You can start by
- Using T-SQL with Warehouses, and get your confidence up in writing stored procedures and functions.
- Using Python and Spark notebooks where appropriate in building ingestion, transformation and orchestration logic, Pandas first then diving into Spark.
You background on BI is desirable - engineering can be transform the data into a desired state, but BI skills are invaluable in turning transformed and cleaned data into insights!
Hi,
thank you for tour advices!
This post can be closed