Forum Discussion

ebjim's avatar
ebjim
Icon for Resolver I rankResolver I
2 years ago
Solved

Shortening sql server import times

I am looking for any suggestions to shorten the import time of one large table, given that in the near term, I'll be using a single node gateway and importing a binary file is not an option.   The ...
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi ebjim 
    If you are importing this into a semantic model,  you could look at incremental refresh, create partitions, review data types, only include columns needed (used for relationships, analysis, measures), verify if you have auto date/time turned off if not being used (this can potentially create very large hidden date time tables for each date time column they are importing), etc.

    Is this historical or incremental load or truncate and load every time? If this is incremental, copy data to files section as LH (sink) does not support merge operation today. A copy activity followed by notebook or Dataflow gen2 is an option.

    Hope this helps. Please let me know if you have any further queries.