Forum Discussion
Date Format Error
Thanks a lot Greg_Deckler ,
If that’s an actual issue, it’s surely worth raising. I will take a look and raise it if needed.
By the way, do you mind looking at my quick question above re possibility of being able to reduce size of excel sheets. That would be great help as transformation of data is taking good amount of time.
Regards,
Gaurav
GK600 - Well, so I guess the first question to ask is if these are all the same format and you are using a Folder query with combine binaries essentially and all the information essentially goes into a single table? That would be optimal since this would maximize the columnar compression of the Vertipaq engine.
Second, turn off the auto-date hierarchy here and refresh (uncheck). Let me know results but this can reduce model size by up to 60% or so:
GK600 - Yes, you should be getting good columnar compression if you are combining binaries into a single fact table.
Thanks again Greg_Deckler,
80% of the data is same format and lies in one of the folder so I think that helps. The columns may be slightly different in each file but I am OK with first file's column headings what it picks automatically (I know I can change that though).
I was already using a Folder query. Looking at the automated steps applied by Power Query, I was not sure whether it expanded binaries column or other record/table colums. So, I manually expanded binaries column now and realised it takes same steps as earlier when expanded at source step below. Hope that makes sense and answer your question.
I wasnt sure but interstingly, Time intelligence was already switched off. See below.
Despite following this, the loading time had been somewhere upto 5-10 minutes. I am OK when I am closing and appying into Power BI desktop as thats one time but I got to simply look at all possible values in a column after a transformation step and it follows the same loading data process.
In other words, it goes through each file like below. Is there a way to know if folder query (binaries) helped reducing size? I am asking because the loading process goes through size of each file (so if a file is 40 mb in the folder, the below screenshot slowly moves up to 40mb and then move to next file.
Thanks very much again for your time to respond.
regards,
Gaurav