Forum Discussion
DBT Orchestration
OK thanks. At the moment you need to run dbt core (and therefore dbt run) on some non-fabric compute unfortunately, like a VM.
You can clone your code from git in the notebook and then run dbt in the notebook.
https://www.linkedin.com/pulse/dbt-microsoft-fabric-my-journ-nghia-tran/Note: Change runtime version in Spark Setting from 1.2 to 1.1 (Runtime version 1.2 does not have git).
Hey AndyDDC . Appreciate you coming back on this. I'm developing on my local machine.
The basic workflow I'd like to have if possible would be like:
- Analytics engineers develop on their local machine.
- Push to github.
- scheduled Fabric pipeline somehow clones the repo.
- Fabric somehow does a dbt run
- Alert messages get emailed ….
One idea I’ve seen floating around is to have a notebook do the lat 3 steps above. I actually pip installed dbt in the notebook, got all excited but then got stuck trying to clone the repo.
You can clone your code from git in the notebook and then run dbt in the notebook.
https://www.linkedin.com/pulse/dbt-microsoft-fabric-my-journ-nghia-tran/
Note: Change runtime version in Spark Setting from 1.2 to 1.1 (Runtime version 1.2 does not have git).
That worked.....nice one Nghia!