Data pipelines and Google Scholar/SERPAPI
Full playlist: https://youtube.com/playlist?list=PLgCWlb4zjb-m7VuIRLjTL9Hw0BSC5J3PP&si=O1c-UZo2rMyvs6MV
Full blog series: https://chatjpt.blog/series/microsoft-fabric-data-edu
Please feel free to leave tips/hints if you spot something I could have done better.
Powerful workflow capabilities are made possible in Microsoft Fabric using data pipelines. For an end-to-end ETL data pipeline, combine a configuration-driven copy activity with your low-code dataflow refresh. Even code-first activities for Spark Notebooks, SQL scripts, stored procedures, and other resources are available. In this video, we will continue to work on a pipeline to ingest Google Scholar JSON data using the SERPAPI. We will insert variables and use expressions to begin to make the pipeline more flexible. Intro to data factory, data pipelines, and dataflow from the Microsoft site: https://learn.microsoft.com/en-us/fabric/data-factory/data-factory-overview