Hi.
We are trying to use the CopyJob to do Merge on some table in order to do incremental loads to a Lakehouse in Fabric from an onpremises SQL Server. Some of the tables we use are not very well constructed so we sometimes have to get rid of some columns in the source for which we use a Query instead of bringing the whole table which sometimes can have more than 100 columns (again not very well constructed table).
On the Copy Activity we have the ability so send a Query or ingest a whole table with all the columns. We would like to see this option available for the CopyJob if at all possible.
Thanks!
4 Comments
- yexuMicrosoft EmployeeStatus changed:NewtoNeeds Votes
- Miwa1Microsoft EmployeeQuery is already supported for the following connectors and we're expanding it to more. Lakehouse AzureSql AzureSqlMI FabricSql SqlServer AzureSQLDatawarehouse Snowflake Google BigQuery Oracle Kusto
- AntonioPinedaRegular VisitorYES! We noticed it is added now. Thanks a lot!
- yexuMicrosoft EmployeeStatus changed:Needs VotestoCompleted
Use query as source has been added in Copy job. See details in Truncate Destination, Queries and Multiple Folders – Simplifying Data Ingestion with Copy Job | Microsoft Fabric Blog | Microsoft Fabric
Recent ideas
Enable mermaid diagrams rendering in Notebooks
Diagrams can be a great explainer of a (Data engineering) process, code or ML process. Notebooks already support including Markdown for adding description. And Mermaid is a well-know way to in...cmaneu5 hours agoMicrosoft EmployeeNeeds Votes3.4KViews52likes5CommentsPower BI Subscriptions enhancements sorely needed
I believe there are several updates needed to Power BI subscriptions: Gate subscription ability. Right now, this is all-on/all-off feature. I would like this feature to be gated for users/securit...OldDogNewTricks13 hours agoKudo CollectorNew689Views26likes3CommentsSearch for models that failed or are in progress
We'd like to be able to search a workspace for models that are currently refreshing or models that have failed to refresh. We have a workspace with a lot of models; and its difficult to identify the...wrjohnston9123813 hours agoRegular VisitorNew10Views1like0CommentsOption to disable or bulk-clear Multitasking session tabs in the Left Navigation Pane
Currently, when working in the Fabric/Power BI Service, opening reports, dataflows, lakehouses, or semantic models automatically stacks them as active session tabs directly under the active workspace...DirkSw15 hours agoNew MemberNew8Views0likes0Comments