How to run a pyspark code directly from a Github Repo?

[lchinelli]

I'd like to create a data pipeline and run a pyspark code directly from a Github repo, is that possible?

[BhaveshPatel]

As far as I know, You can not run programmimg code such as Pyspark from Github repo. It is for CI/CD ( Github Repo) . By the way, why you have to do this. 

Rather than I should use Dataflow Gen 2 or Python Notebooks. 

[KevinChant]

Do you mean run a notebook from a GitHub repo using a GitHub workflow? If so then absolutely.

I did a post that shows how you can do it with Azure DevOps, you can port the logic over:
https://www.kevinrchant.com/2025/01/31/authenticate-as-a-service-principal-to-run-a-microsoft-fabric...