Hi.
We are trying to use the CopyJob to do Merge on some table in order to do incremental loads to a Lakehouse in Fabric from an onpremises SQL Server. Some of the tables we use are not very well constructed so we sometimes have to get rid of some columns in the source for which we use a Query instead of bringing the whole table which sometimes can have more than 100 columns (again not very well constructed table).
On the Copy Activity we have the ability so send a Query or ingest a whole table with all the columns. We would like to see this option available for the CopyJob if at all possible.
Thanks!
4 Comments
- yexuMicrosoft EmployeeStatus changed:NewtoNeeds Votes
- Miwa1Microsoft EmployeeQuery is already supported for the following connectors and we're expanding it to more. Lakehouse AzureSql AzureSqlMI FabricSql SqlServer AzureSQLDatawarehouse Snowflake Google BigQuery Oracle Kusto
- AntonioPinedaRegular VisitorYES! We noticed it is added now. Thanks a lot!
- yexuMicrosoft EmployeeStatus changed:Needs VotestoCompleted
Use query as source has been added in Copy job. See details in Truncate Destination, Queries and Multiple Folders – Simplifying Data Ingestion with Copy Job | Microsoft Fabric Blog | Microsoft Fabric
Recent ideas
OneLake Catalog full-text and content search
Why is it not possible to search OneLake by item description or even content? So I was thinking about how a user can find a table if they don't know in which lakehouse it is located? Why can't you se...TePe4 hours agoHelper IVPlanned1.1KViews21likes5Comments