Forum Discussion

michaelfabric's avatar
michaelfabric
Frequent Visitor
2 months ago
Solved

dbt job pipeline activity: bug

Hi everone,   The operation (like run, build and compile, under "Advanced Settings") of a dbt job activity in a pipeline is not being pushed to the repo when doing a commit. As a result, my fabric-...
data engineering
data pipeline
  • michaelfabric's avatar
    michaelfabric
    1 month ago

    I created a support ticket and have the following update to share: Microsoft support has excalated this issue and a fix will be implemented:

    "We have done further tests on this internally and were able to reproduce the behavior. Then escalated it internally to our Product team. As a result, this will be addressed and the current tentative for the fix is in September. 

    If things go as planned, it should be during the first week of September but deployment can vary depending on different regions."

     

    Please check the reply of ryan-schofield for a temporary workaround.

sannavajjala's avatar
sannavajjala
Icon for Resolver II rankResolver II
1 month ago

Hi,

 

Hi Michael,

This sounds like a product limitation or serialization issue with the current Fabric Git integration rather than something you're doing incorrectly. If the Operation setting (Run, Build, Compile) is configured in the UI but is not persisted in the repository artifacts, deployments through Fabric CI/CD can result in an invalid activity definition in the target workspace.

The fact that manually adding "operation": "build" to the pipeline JSON resolves the runtime error, but Fabric subsequently removes it during the next commit, suggests the Git representation of the dbt activity is not fully round-tripping all properties. In that case, the source of truth in the UI and the source-controlled artifact are becoming inconsistent.

A few things I'd recommend checking:

  • Verify all environments are running the same Fabric feature set and Git integration experience.
  • Confirm whether the issue occurs with newly created dbt activities or only existing ones.
  • Compare the JSON generated before and after changing the Operation value to see whether any related properties are being persisted.
  • Test deploying the pipeline directly between workspaces without Git to determine whether the issue is specifically tied to source control serialization.

If the behavior is reproducible, it is likely a bug and worth raising with Microsoft Support. Be sure to include:

  • The pipeline JSON stored in Git
  • Screenshots of the Operation setting in the UI
  • Deployment logs showing the missing Operation error
  • The Fabric workspace and Git integration experience being used

In the meantime, if Fabric continues to remove the property during commits, any manual JSON edits are unlikely to be a sustainable workaround because the platform will overwrite them during synchronization. Hopefully others in the community can confirm whether they're seeing the same behavior with dbt activities and Git-managed deployments.

 

Thanks,

Manoj Annavajjala