Forum Discussion
Extracting data from External API
- 5 months ago
Hi SaumyaMathai ,
For most Fabric medallion implementations, the recommended approach is usually closest to Option 2.
External APIs are typically ingested into the same Bronze layer as raw data, since Bronze is designed to store source data in its original form regardless of where it comes from (SQL, APIs, files, etc.). From there, the data can follow the same Bronze > Silver > Gold transformation pattern as the rest of your pipeline.
To keep things maintainable when you have many APIs, a common practice is to use separate ingestion pipelines per source system, but land the raw outputs into the shared Bronze layer often partitioned or organized by source system. This avoids maintaining multiple medallion architectures while still keeping ingestion logic modular.
Creating an additional Platinum layer is generally not necessary unless you have a very specific consumption or governance requirement beyond the standard Gold layer.
Overall, a single medallion architecture with source-specific ingestion pipelines feeding Bronze tends to scale well and keeps the architecture simpler to manage.
Hope this helps.
Thank you. - 5 months ago
Hello SaumyaMathai,
Ingest external API data into your existing Bronze → Silver → Gold pipeline. Store raw API responses in Bronze, normalize and clean in Silver, and integrate with internal SQL data in Gold. Use incremental loads, error handling, and schema tracking to keep pipelines scalable and maintainable.
Refs:
Hi SaumyaMathai ,
Thank you Olufemi7 , deborshi_nag for your inputs.
We’d like to follow up regarding the recent concern. Kindly confirm whether the issue has been resolved, or if further assistance is still required. We are available to support you and are committed to helping you reach a resolution.
Thank you.
Hi , I have got an idea on how to move forward now . Thanks so much for all your inputs and suggestions, they have been really helpful.
Appreciate the support from this community.