Copy Job - Lakehouse Files to Lakehouse Table
Hi there, I am exactly trying to achieve what Patrick is doing in the video: Nutzen Sie den Kopierauftrag, um die neueste Datei in die Data Factory zu kopieren. - YouTube Shortcut loading Blob Files into Lakehouses Files and then using a Copy Job to move in the same Lakehouse with incremental load into the Table section. However this seems to be not working anymore? I am receiving a error message stating that: "ErrorCode=ApplyChangesNotSupported,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ApplyChanges is not supported for the copy pair from Lakehouse to LakehouseTable.,Source=Microsoft.DataTransfer.ClientLibrary,'" However I am curious if there is anything that I am doing wrong? Are there better ways to move within a Lakehouse from Files into Table incrementally with a low-code experience? Thanks for the help!
Incremental Copy Job error from NetSuite
I am trying to configure a copy job to incrementally load data from NetSuite SuiteAnalytics Connect. The source is accessed using the NetSuite ODBC driver installed on an on-prem data gateway. I receive this error when trying to run the job, regardless of table selected: Failure happened on 'Source' side. ErrorCode=UserErrorOdbcOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=ERROR [HY000] [NetSuite][ODBC 64bit driver][OpenAccess SDK SQL Engine]Error getting the literal value of right operand.[10169],Source=Microsoft.DataTransfer.Runtime.GenericOdbcConnectors,''Type=System.Data.Odbc.OdbcException,Message=ERROR [HY000] [NetSuite][ODBC 64bit driver][OpenAccess SDK SQL Engine]Error getting the literal value of right operand.[10169],Source=NQoa27.dll,' The field used as a watermark is recognised by Fabric as a datetime column. Before I raise a support request with MS, has anyone else encountered this issue and found a solution?
Copy Job Not Saving
I want to know how everyone is dealing with this. I have a Copy Job that brings over 30+ tables from one lakehouse to another and has column mappings. I find that when I try to edit the mapping of one table after running the Copy Job and then hitting the Apply button, I receive a validation model error message even though I only edited a single table and then the mappings of the other tables disappear and I have to redo the mappings for all 30+ tables. If I try to add another table from the source, all my mappings from the previous sources will disappear as well and I have to redo it. I was hoping that I could hold onto the JSON code and paste in the old mappings but there's no way to edit the Copy Job JSON file only to view it. How is everyone editing their Copy Jobs?
Copy Data from Blob in data pipeline occurring Failed to load
HI experts, I am trying to instead my data platform with Microsoft Fabric. I was setup vNet data gateway and checked vNDG online and tested connectivity to Blob. Blob was setup PE in A subnet and vNDG located B subnet. There is Secured vHub and force routing to AzFW. AzFW Policy allowed vNDG HTTPS access to '*.blob.core.windows.net' in application rules. Checked logs of AzFW show traffic was passed to PE and logs of Blob show vNDG sussccess to communicate Blob. But I can't browse blob in the copy data activity. Click test connection in properties of copy data it shows May I know how to troubleshoot it? Can any tools view logs following Activity ID?
Cannot merge data from a CDC enabled table
Hi, I'm trying to upsert data from a CDC enabled table in an Azure SQL database table to a Fabric database using a copy job. As you can see in the screenshot below, I have selected Merge as the update method. However, the column mapping is disabled when CDC is enabled - which is expected. There's a primary key in the dbo.customer table in the source. However, copy job isn't recognizing it nor allowiing me to specify it. Is this a known limitation or a bug?
