Forum Discussion
Excel sheets from sharepoint
Hello vpanchu
i know that if you are reading from so many files and then want to add columns in your productive systems, this will be really ugly, because however to update your preview PQ has to start getting the data again. To it would make sense so reduce the amount of data when your are developing by using two different parameters so you can work on a small amount of data and when it's finished you can use all your data. This is how I did always when working with a lot of files
BR
Jimmy
Hello vpanchu
reading a lot of files will takes some time, thats for sure. But it should not be hours, but within 15 - 20 minutes. You didn't tell me how long it takes that you are concered about performance.
about your second question. Didn't you say that all files are the same? If yes, then you have to remove the top empty rows of all files before promote headers. If no, you need a logic to work around this issue. Some characteristics that you can query and then continue accordingly. You can apply my technique.. indipendingly what transformation you have to to beforehand. Lets say cut 2 empty rows and then promote headers... then you get a table where you can use the columnnames to make a comparison with your template and check wheter something is wrong or not
If this post helps or solves your problem, please mark it as solution (to help other users find useful content and to acknowledge the work of users that helped you)
Kudoes are nice too
Have fun
Jimmy
Hi Jimmy801 basically this paticular app taks atleast 15 to 25 min to open up when i try to open the PBIX file thats forst issue and refreshing takes almost 20 min , to add some additional query transformation it lags and takes ages to reflect ,for examaple if i add a conditional column or delete a column it takes 5 to 10 minutes to reflect the change and it slows down the developement speed , just wanted to understand if its because i am getting data from 40 share point folders ?
the second option you have suggested i shall try that as well ,thank you for the help.
Regards
VP
Hello vpanchu
i know that if you are reading from so many files and then want to add columns in your productive systems, this will be really ugly, because however to update your preview PQ has to start getting the data again. To it would make sense so reduce the amount of data when your are developing by using two different parameters so you can work on a small amount of data and when it's finished you can use all your data. This is how I did always when working with a lot of files
BR
Jimmy
thank you jimmy will try it out.
regards
VP