Forum Discussion

PowerBIU10's avatar
PowerBIU10
New Member
4 years ago
Solved

Azure Data Lake Gen2 CDM folder refresh times

Hey all,   We've recently configured Azure Synapse Link for Dataverse to export all selected tables to a Data Lake Gen2 storage (not using Azure Synapse Workspaces, data goes straight from Datavers...
Need Help
  • stuartreid's avatar
    stuartreid
    4 years ago

    Hello.

     

    Don't know if you got an answer to this one yet... but I'll give it a shot.

    When you're using the CDM folders to pull in data from the datalake, it's actually using the snapshot and not reading through all the individual files.

    You're snapshot is probably being updated every 30 minutes.

     

    Hope that helps.

Anonymous's avatar
Anonymous
Not applicable
4 years ago

Hi PowerBIU10 ,

 

Please check this blog, it may be helpful to you. The conclusions inside introduce the reasons for the slowness.

 

Best Regards,

Stephen Tao

 

If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

PowerBIU10's avatar
PowerBIU10
New Member
4 years ago

Thank you! 

 

Though I am looking at why the CDM folder itself takes time to reflect changes in the ADLS?

The CSVs have been added new data, but the CDM does not show these changes for 30 minutes to 1 hour.

 

Thanks again,