Forum Discussion
Helper IVPipeline Naming Convention
Hi,
I'm curious to know of naming conventions used.
I have 1 ETL ingest config table which has entries for various sources and tables.
Fabric copy activity being dynamic, i can parametrize any source any number of tables as long connections are established.
Sources are like ERP systems, CRM systems, sftp files, SQL server) all go into 1 fabric warehouse or lakehouse.
Set up is 1 parent pipeline doing a look up on etl table, for each loop invoking ONE child pipeline extracting data from various sources based on dynamic connection. How to pair names?
Ingest_Parent --> Ingest_child?
GetSources --> ProcessSources?
Ingest_Master - > inner pipeline is slave? ( i dont want to use master or slave like how harddrives )
Ingest_Primary --> can name it as ingest_secondary?
Ingest_Main --> sub?
Im sorry im bad at naming.. Any better way?
Probably the hardest part of engineering 🙂
A good naming sould be functional, scalable, and self-documenting.
Do you have any convention name document in your company ?
Some example :
You can use prefixes for items, use the role to describ and the verb for understanding the action.
<object>_<role>_<verb>
like :
- pl_orchestrate_ingestion (parent-pipeline)
- pl_execute_ingestion (child)
pl_ for pipelines, df_ for dataflow, nb_ for notebooks.
4 Replies
- AntoineW
Super User
Probably the hardest part of engineering 🙂
A good naming sould be functional, scalable, and self-documenting.
Do you have any convention name document in your company ?
Some example :
You can use prefixes for items, use the role to describ and the verb for understanding the action.
<object>_<role>_<verb>
like :
- pl_orchestrate_ingestion (parent-pipeline)
- pl_execute_ingestion (child)
pl_ for pipelines, df_ for dataflow, nb_ for notebooks.
v-echaithraCommunity Support
Hi AJAJ ,
Thank you for reaching out to Microsoft Community.For metadata-driven ingestion scenarios, it’s generally recommended to name pipelines based on their functional role rather than their position in a hierarchy.
In Fabric, this typically results in two logical roles:
An orchestration pipeline responsible for resolving metadata, handling control flow, and managing iteration
An execution pipeline responsible for performing the actual copy or ingestion activitiesUsing neutral, role-based names helps keep this distinction clear and scales well as solutions evolve. For example:
Ingest_Orchestrator
Ingest_ExecutorThis approach avoids hierarchical terminology (such as parent/child) and remains intuitive if additional pipelines or source-specific executors are introduced later.
Hope this helps.
Thank you.- v-echaithra
