<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Support for append mode in CDC in copy job in Data Engineering</title>
    <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4873082#M13396</link>
    <description>&lt;P&gt;Hi &lt;a href="javascript:void(0)" data-lia-user-mentions="" data-lia-user-uid="1134405" data-lia-user-login="kjetil_hamre" class="lia-mention lia-mention-user"&gt;kjetil_hamre&lt;/a&gt;&amp;nbsp;,&lt;/P&gt;&lt;P&gt;You’re not doing anything wrong this is actually a &lt;STRONG&gt;current platform limitation&lt;/STRONG&gt; in Microsoft Fabric.&lt;/P&gt;&lt;P&gt;Right now, &lt;STRONG&gt;Change Data Capture (CDC)&lt;/STRONG&gt; in Fabric &lt;STRONG&gt;only supports destinations that use the “Merge” update method&lt;/STRONG&gt;.&lt;BR /&gt;CDC with &lt;STRONG&gt;Append&lt;/STRONG&gt; destinations, &lt;STRONG&gt;Lakehouse&lt;/STRONG&gt;, or &lt;STRONG&gt;Fabric SQL Database&lt;/STRONG&gt; is &lt;STRONG&gt;not yet supported&lt;/STRONG&gt;. That’s why you’re getting those errors when you try to set up CDC → Append or CDC → Lakehouse.&lt;/P&gt;&lt;P&gt;If your goal is to build &lt;STRONG&gt;SCD Type 2&lt;/STRONG&gt; history (to keep inserts, updates, and deletes), here’s the recommended workaround:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;&lt;P&gt;&lt;STRONG&gt;Use CDC → Merge&lt;/STRONG&gt; to land changes into a &lt;STRONG&gt;staging table&lt;/STRONG&gt; (for example, in Azure SQL or Fabric Warehouse).&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Then use a &lt;STRONG&gt;Fabric notebook or Dataflow&lt;/STRONG&gt; to read from that staging table and apply your &lt;STRONG&gt;SCD Type 2 logic&lt;/STRONG&gt; —&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Insert new records.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Update existing rows by setting an end_date or is_current flag.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Handle deletes accordingly.&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Finally, write the SCD-2 history data into your &lt;STRONG&gt;Lakehouse (append mode)&lt;/STRONG&gt; as a separate layer.&lt;/P&gt;&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;This is the approach most teams are following until Fabric adds direct &lt;STRONG&gt;CDC support for Lakehouse and Append destinations&lt;/STRONG&gt;, which is already on Microsoft’s roadmap.&lt;/P&gt;&lt;P&gt;So in short — it’s not supported &lt;EM&gt;yet&lt;/EM&gt;, and your setup is correct. Use CDC → Merge + Notebook for SCD-2 logic as a reliable solution for now.&lt;/P&gt;&lt;P&gt;Hope this helps!&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Gopi Krishna&lt;/STRONG&gt;&lt;/P&gt;</description>
    <pubDate>Wed, 12 Nov 2025 11:04:05 GMT</pubDate>
    <dc:creator>Ugk161610</dc:creator>
    <dc:date>2025-11-12T11:04:05Z</dc:date>
    <item>
      <title>Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4866920#M13261</link>
      <description>&lt;P&gt;Hi im trying out CDC from a source SQL database. Really, what I need to have a stream of all the inserts, updates and deletes in the source database. With such a stream I can create SCD-2 records in a medallion dataplatform. It seems CDC with an "append" destination should solve this. I have tried to set it up with copy-job into an Azure SQL managed instance. However - the wizard stops with this error:&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;&lt;STRONG&gt;CDC is only supported for destinations where the update method is merge. For all other update methods, please use watermark-based incremental copy, which requires specifying an incremental column to identify changes from the source.&lt;/STRONG&gt;&lt;BR /&gt;&lt;BR /&gt;I thought that was a very weird error message. I mean it should work? watermark-based incremental copy is not an option as the source does not have a "last updated" column - AND it would not capture deletes.&lt;BR /&gt;&lt;BR /&gt;Really I wanted to do it right into a lakehouse - but that fails with:&lt;BR /&gt;&lt;SPAN&gt;CDC is not supported for Microsoft Fabric Lakehouse Table as destinations. Please use watermark-based incremental copy, which requires specifying an incremental column to identify changes from the source.&lt;/SPAN&gt;&lt;BR /&gt;&lt;BR /&gt;To do it into a fabric sql server fails with:&lt;BR /&gt;&lt;SPAN&gt;CDC is not supported for SQL Database as destinations. Please use watermark-based incremental copy, which requires specifying an incremental column to identify changes from the source.&lt;/SPAN&gt;&lt;BR /&gt;&lt;BR /&gt;So, I wonder is this functionality that is not yet supported and is coming very soon, or am I just looking in the wrong place? Any help apprechiated.&lt;BR /&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 05 Nov 2025 11:39:22 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4866920#M13261</guid>
      <dc:creator>kjetil_hamre</dc:creator>
      <dc:date>2025-11-05T11:39:22Z</dc:date>
    </item>
    <item>
      <title>Re: Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867398#M13272</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="javascript:void(0)" data-lia-user-mentions="" data-lia-user-uid="1134405" data-lia-user-login="kjetil_hamre" class="lia-mention lia-mention-user"&gt;kjetil_hamre&lt;/a&gt;, when testing your scenarios I get the same warnings on all destinations. So I think you are looking in the right place, but maybe it's still too early. Have you check the Know limitations section on the CDC page?&amp;nbsp;&lt;A href="https://learn.microsoft.com/en-us/fabric/data-factory/cdc-copy-job#known-limitations" target="_self"&gt;Change data capture (CDC) in Copy Job - Known limitations&lt;/A&gt;&lt;BR /&gt;&lt;BR /&gt;Hope this helps. If so, please give a Kudos &lt;span class="lia-unicode-emoji" title=":thumbs_up:"&gt;👍&lt;/span&gt; and mark as Accepted Solution &lt;span class="lia-unicode-emoji" title=":heavy_check_mark:"&gt;✔️&lt;/span&gt;.&lt;/P&gt;</description>
      <pubDate>Wed, 05 Nov 2025 23:40:51 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867398#M13272</guid>
      <dc:creator>nielsvdc</dc:creator>
      <dc:date>2025-11-05T23:40:51Z</dc:date>
    </item>
    <item>
      <title>Re: Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867756#M13285</link>
      <description>&lt;P&gt;It does not really help, but I gave you Kudos anyway &lt;span class="lia-unicode-emoji" title=":slightly_smiling_face:"&gt;🙂&lt;/span&gt; Maybe someone else have more insight?&lt;/P&gt;</description>
      <pubDate>Thu, 06 Nov 2025 08:19:26 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867756#M13285</guid>
      <dc:creator>kjetil_hamre</dc:creator>
      <dc:date>2025-11-06T08:19:26Z</dc:date>
    </item>
    <item>
      <title>Re: Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867833#M13288</link>
      <description>&lt;P&gt;someone pointed me to this!&amp;nbsp;&lt;A href="https://community.fabric.microsoft.com/t5/Fabric-Ideas/All-CDC-Changes-to-be-captured-in-Copy-Job-s-Incremental-Copy/idc-p/4867782#M164874" target="_blank"&gt;All CDC Changes to be captured in Copy Job's Incre... - Microsoft Fabric Community&lt;/A&gt;&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;I do not understand - documentation seems to say this is not just an idea? And the idea has not got any votes apart from mine? I am the only person living in this universe?&lt;/P&gt;</description>
      <pubDate>Thu, 06 Nov 2025 09:17:51 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4867833#M13288</guid>
      <dc:creator>kjetil_hamre</dc:creator>
      <dc:date>2025-11-06T09:17:51Z</dc:date>
    </item>
    <item>
      <title>Re: Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4871148#M13343</link>
      <description>&lt;P&gt;Hi&amp;nbsp;&lt;a href="javascript:void(0)" data-lia-user-mentions="" data-lia-user-uid="1134405" data-lia-user-login="kjetil_hamre" class="lia-mention lia-mention-user"&gt;kjetil_hamre&lt;/a&gt;&amp;nbsp;&lt;BR /&gt;&lt;BR /&gt;&lt;/P&gt;
&lt;P&gt;I completely understand your concern, and you’re not the only one encountering this scenario. Currently, CDC in Copy Jobs is supported only for destinations where the update method is Merge. As mentioned in the documentation’s Known limitations section, CDC isn’t yet available for destinations such as Lakehouse or SQL Database, which is why you’re seeing the warnings.&lt;/P&gt;
&lt;P&gt;Submitting this as an Idea was the right approach&amp;nbsp; the Product Group team reviews these requests regularly, and prioritization is largely influenced by customer votes and usage impact. I’d recommend adding more context about your use case in this thread, as it helps the team better assess the feature’s importance.&lt;/P&gt;</description>
      <pubDate>Mon, 10 Nov 2025 14:36:34 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4871148#M13343</guid>
      <dc:creator>v-aatheeque</dc:creator>
      <dc:date>2025-11-10T14:36:34Z</dc:date>
    </item>
    <item>
      <title>Re: Support for append mode in CDC in copy job</title>
      <link>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4873082#M13396</link>
      <description>&lt;P&gt;Hi &lt;a href="javascript:void(0)" data-lia-user-mentions="" data-lia-user-uid="1134405" data-lia-user-login="kjetil_hamre" class="lia-mention lia-mention-user"&gt;kjetil_hamre&lt;/a&gt;&amp;nbsp;,&lt;/P&gt;&lt;P&gt;You’re not doing anything wrong this is actually a &lt;STRONG&gt;current platform limitation&lt;/STRONG&gt; in Microsoft Fabric.&lt;/P&gt;&lt;P&gt;Right now, &lt;STRONG&gt;Change Data Capture (CDC)&lt;/STRONG&gt; in Fabric &lt;STRONG&gt;only supports destinations that use the “Merge” update method&lt;/STRONG&gt;.&lt;BR /&gt;CDC with &lt;STRONG&gt;Append&lt;/STRONG&gt; destinations, &lt;STRONG&gt;Lakehouse&lt;/STRONG&gt;, or &lt;STRONG&gt;Fabric SQL Database&lt;/STRONG&gt; is &lt;STRONG&gt;not yet supported&lt;/STRONG&gt;. That’s why you’re getting those errors when you try to set up CDC → Append or CDC → Lakehouse.&lt;/P&gt;&lt;P&gt;If your goal is to build &lt;STRONG&gt;SCD Type 2&lt;/STRONG&gt; history (to keep inserts, updates, and deletes), here’s the recommended workaround:&lt;/P&gt;&lt;OL&gt;&lt;LI&gt;&lt;P&gt;&lt;STRONG&gt;Use CDC → Merge&lt;/STRONG&gt; to land changes into a &lt;STRONG&gt;staging table&lt;/STRONG&gt; (for example, in Azure SQL or Fabric Warehouse).&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Then use a &lt;STRONG&gt;Fabric notebook or Dataflow&lt;/STRONG&gt; to read from that staging table and apply your &lt;STRONG&gt;SCD Type 2 logic&lt;/STRONG&gt; —&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Insert new records.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Update existing rows by setting an end_date or is_current flag.&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Handle deletes accordingly.&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Finally, write the SCD-2 history data into your &lt;STRONG&gt;Lakehouse (append mode)&lt;/STRONG&gt; as a separate layer.&lt;/P&gt;&lt;/LI&gt;&lt;/OL&gt;&lt;P&gt;This is the approach most teams are following until Fabric adds direct &lt;STRONG&gt;CDC support for Lakehouse and Append destinations&lt;/STRONG&gt;, which is already on Microsoft’s roadmap.&lt;/P&gt;&lt;P&gt;So in short — it’s not supported &lt;EM&gt;yet&lt;/EM&gt;, and your setup is correct. Use CDC → Merge + Notebook for SCD-2 logic as a reliable solution for now.&lt;/P&gt;&lt;P&gt;Hope this helps!&lt;/P&gt;&lt;P&gt;&lt;STRONG&gt;Gopi Krishna&lt;/STRONG&gt;&lt;/P&gt;</description>
      <pubDate>Wed, 12 Nov 2025 11:04:05 GMT</pubDate>
      <guid>https://community.fabric.microsoft.com/t5/Data-Engineering/Support-for-append-mode-in-CDC-in-copy-job/m-p/4873082#M13396</guid>
      <dc:creator>Ugk161610</dc:creator>
      <dc:date>2025-11-12T11:04:05Z</dc:date>
    </item>
  </channel>
</rss>

