Forum Discussion
Solved
Ingest parquet files in SQL Azure
Hi We are exporting data from D365 to a Gen 2 datalake and from there I want to ingest it into a SQL azure database. I tried using the " Copy Dataverse data into Azure SQL using Synapse Link" te...
Hello alexp01482
I would suggest this approach- Use Azure Synapse's serverless SQL pool to directly ( OPENROWSET) query Parquet files stored in Azure Data Lake Gen2.
- Transform the data using T-SQL and load it into Azure SQL Database using pipelines or external tables.
- This method is highly efficient for large datasets due to parallel processing and minimal data movement
When exporting data from Dynamics 365 to Azure Data Lake Gen2, it often adheres to the Common Data Model (CDM).
If this is helpful , please accept the answer and give kudos
Hello alexp01482
I would suggest this approach
- Use Azure Synapse's serverless SQL pool to directly ( OPENROWSET) query Parquet files stored in Azure Data Lake Gen2.
- Transform the data using T-SQL and load it into Azure SQL Database using pipelines or external tables.
- This method is highly efficient for large datasets due to parallel processing and minimal data movement
When exporting data from Dynamics 365 to Azure Data Lake Gen2, it often adheres to the Common Data Model (CDM).
If this is helpful , please accept the answer and give kudos