Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.
Hi
I have to load and combine txt files that each have multiple CSV formatted blocks. Something like:
# Some header info1
# Some header info2
#@startFirstBlock
A;B;C;D
E;F;G;H
I;L;M;N
#@endFirstBlock
#@startSecondBlock
1;2;3
4;5;6
#@endSecondBlock
As I need only the " header" rows and the ones between "#@startFirstBlock" and "#@endFirstBlock", is there a way to only load these rows skipping the rest during the loading process?
My question/problem is essentially performance driven: I have 100s of files with 1000s of rows in each "block" and each file can contain 10 and more of these blocks. To load all is taking verrrrrry long and once loaded I have to delete the rows from the not needed blocks (that also takes time). Is there a way to just load what I need from the start?
Thanks for helping.
Solved! Go to Solution.
Hi @Anonymous ,
no, that's not possible. These kind of files have to be ingested fully before anything can be extracted from them.
But it could be that you are making expensive transformations on them. Check out my performance tips for traps to avoid: Speed/Performance aspects – The BIccountant
Imke Feldmann (The BIccountant)
If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!
How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries
Hi @Anonymous ,
no, that's not possible. These kind of files have to be ingested fully before anything can be extracted from them.
But it could be that you are making expensive transformations on them. Check out my performance tips for traps to avoid: Speed/Performance aspects – The BIccountant
Imke Feldmann (The BIccountant)
If you liked my solution, please give it a thumbs up. And if I did answer your question, please mark this post as a solution. Thanks!
How to integrate M-code into your solution -- How to get your questions answered quickly -- How to provide sample data -- Check out more PBI- learning resources here -- Performance Tipps for M-queries
Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City
Check out the April 2024 Power BI update to learn about new features.
User | Count |
---|---|
110 | |
94 | |
80 | |
67 | |
59 |
User | Count |
---|---|
150 | |
119 | |
104 | |
87 | |
67 |