Forum Discussion
Data engg career
Hello everyone,
I'm currently pursuing a Master's in Data Science and am working toward a career in data engineering. As AI continues to reshape the industry, I'm interested in understanding how aspiring data engineers can prepare for the future and build skills that remain valuable over the long term.
Although I'm still a student, I've been actively developing hands-on experience through personal and academic projects. So far, I've worked with technologies including:
Python
SQL
PySpark
Microsoft Fabric
Azure Data Factory
Azure Databricks
Snowflake
Azure SQL
Power BI
Delta Lake
ETL pipelines
Git/GitHub
Most of my projects involve building end-to-end data pipelines, working with lakehouse architectures, and using cloud-based data platforms. I'm also exploring how AI is influencing modern data engineering workflows.
I'd love to hear from professionals already working in the field:
If you were beginning your data engineering journey today, what would you prioritize learning over the next couple of years?
Which technical and non-technical skills do you believe will have the greatest impact in the age of AI?
How do platforms like Microsoft Fabric, Azure, and Databricks compare with core data engineering fundamentals in terms of long-term career value?
What types of projects or portfolio work tend to catch the attention of recruiters and hiring managers?
Looking back, what advice or lessons would you share with someone preparing for their first data engineering role?
I appreciate any insights, recommendations, or experiences you're willing to share. Thank you!
First off, well done. I'm getting some real use case and practical work experience whilst learning. The biggest thing for me? When looking on how to. Work in a career is the first thing is always be willing to learn new things constantly this environment is always changing and there are always new things to learn, so you cannot keep still and you just gotta keep relearning constantly.
What I would also say. Is that the fundamentals still apply. Learning how to understand SQL build, dimension and fact tables still apply. Everything always comes back into a table format no matter what the source data is. So make sure you know and understand those concepts really well because the everything always comes back to the fundamentals.And in terms of what people are looking for, when? Hiring first is around the willingness to learn because everyone can learn something new. The attitude in terms of being wanting to learn from others and often just listening to what others have to say is another big plus as well as. Making sure that you can often solve really hard problems doesn't always have to be solved immediately or quickly, but you have a way of understanding what people are asking for and then going through and solving the problem. I think that could really help.
1 Reply
First off, well done. I'm getting some real use case and practical work experience whilst learning. The biggest thing for me? When looking on how to. Work in a career is the first thing is always be willing to learn new things constantly this environment is always changing and there are always new things to learn, so you cannot keep still and you just gotta keep relearning constantly.
What I would also say. Is that the fundamentals still apply. Learning how to understand SQL build, dimension and fact tables still apply. Everything always comes back into a table format no matter what the source data is. So make sure you know and understand those concepts really well because the everything always comes back to the fundamentals.And in terms of what people are looking for, when? Hiring first is around the willingness to learn because everyone can learn something new. The attitude in terms of being wanting to learn from others and often just listening to what others have to say is another big plus as well as. Making sure that you can often solve really hard problems doesn't always have to be solved immediately or quickly, but you have a way of understanding what people are asking for and then going through and solving the problem. I think that could really help.