Power BI is turning 10! Tune in for a special live episode on July 24 with behind-the-scenes stories, product evolution highlights, and a sneak peek at what’s in store for the future.
Save the dateEnhance your career with this limited time 50% discount on Fabric and Power BI exams. Ends August 31st. Request your voucher.
Hello
I'm on a MC fabric project and i use notebook to concatenate severals csv. I would have an other way to do this.
I want ton use pipeline and the get metadata activity.
The folder architecture is on 3 folder level.
Example
Parent folder / child folder / csv
How should i use a pipeline and a for each activity to get the metadata and could copy the csv ?
Thanks for help
Solved! Go to Solution.
Hi @Bendelyon,
In that case you could also use a Get metadata activity before the for each to retrieve your structure first and use that as items in your for each loop instead of the variable I proposed.
Here is a YouTube video which shares such an approach: https://www.youtube.com/watch?v=EU6Ex9EhWyA
Hi @Bendelyon,
With in the for each activity you can use an array. With in the for each you can reference to items of that array via the Dynamic content. A reference would be:
@item()
Below is an example to demonstrate this with the Get metadata activity
1. Example Folder structure
2. Array variable
3. Usage of variable in For Each Items property
@variables('CSVArray')
4. Reference to item of array in GetMetadata activity
@item()
5. Test run pipeline
Hello
Thanks for your help and for five me the "logical" to think.
However i have many parent folder.
Example
Parent folder a / child folder a / csv
Parent folder b / child folder a / csv
Etc
I need to iterate on thé parent folder to enter on the child folder ans get the csv whitch is on.
Thanks for your help
Hi @Bendelyon,
In that case you could also use a Get metadata activity before the for each to retrieve your structure first and use that as items in your for each loop instead of the variable I proposed.
Here is a YouTube video which shares such an approach: https://www.youtube.com/watch?v=EU6Ex9EhWyA
Hi @Bendelyon ,
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet .
Otherwise, will respond back with the more details and we will try to help .
Hi @Bendelyon ,
Glad to know that your query got resolved. Please continue using Fabric Community on your further queries.
Do you mean to concatenate all the CSV files into a single CSV file ?
Then the below blog acts as a sample reference :
https://www.c-sharpcorner.com/article/merge-multiple-json-files-via-synapse-data-factory-pipelines/
wherein you can use the Copy activity recursive and merge into single property
Hello
My principal problem is to get metadata to navigate onto the folder structure.
Thanks
User | Count |
---|---|
25 | |
17 | |
6 | |
5 | |
2 |
User | Count |
---|---|
50 | |
43 | |
18 | |
7 | |
6 |