Forum Discussion
Solved
Optimizing data model update time
Hello, What kind of recomendation coul be to reduce model update time? I've got folders with > 30Gb of CSV data, and each month I add +2Gb. Last try of update take about 2 hour, but it take ...
Cool. The dataflow serves as intermediate data source. Multiple datasets can now consume from the dataflow instead of recreating the import from files. And yes you need to refresh the dataset, now that your dataflow is updated. But see, even reading from a hyperfast single table Azure source, it takes 20min to refresh, that´s the bare minimum under ideal conditions. If I calculated correctly that equals ~200Mbps! on a shared, free ressource! Therefore, if you find a 60min solution reading from text files that need to be processed row per row, I think that should be acceptable. If that solves your problem, please accept as solution to close the case. Alexander
