CosmosDB Vs Azure AI Search

Dear All,

We are building a modern data platform based on Microsoft Fabric and Purview, designed to serve as a proper Data Marketplace capable of handling structured, semi-structured, and unstructured data.

The reason for this message concerns specifically the unstructured data workstream and how best to architect it end-to-end.

Context and current design

We do not yet have the full set of use cases needed to finalise the E2E design with confidence. However, we have already defined the ingestion and bronze layer architecture as follows:

1. A third-party tool (selected by the business) extracts source files and converts each one into a JSON payload containing both the file metadata and its content.
2. These payloads are published to Azure Service Bus.
3. A triggered pipeline collects the payloads from the bus and lands them in our Landing Zone, which consists of a dedicated Fabric workspace containing a Lakehouse.
4. The JSON files are stored immutably in the Lakehouse Files section.
5. Using the same pipeline and trigger, a notebook processes each JSON file and splits it into two Delta tables in the Bronze layer — one for metadata, one for content.

We are comfortable with this design up to and including Bronze.

The open question — Silver onwards

This is where we need your input. Two proposals are on the table:

1. Use a Cosmos DB (as a Fabric item) in a dedicated Gold workspace as a document serving layer.
2. Introduce Azure AI Search, leveraging its built-in chunking and embedding capabilities to avoid having to implement those steps ourselves in Silver.

We see three possible architecture options :

• Option 1 — Cosmos DB only, serving both as document store and RAG backend. This would require us to implement chunking and embeddings ourselves (likely in Silver or during ingestion into Cosmos DB).
• Option 2 — Cosmos DB as document store + Azure AI Search for RAG and semantic search. In this case, chunking and embeddings would be handled by Azure AI Search, not by us.
• Option 3 — Azure AI Search only, handling document storage, chunking, embeddings, and RAG in a single service.

For options 2 and 3, we believe Azure AI Search's native indexing capabilities would eliminate the need for us to build chunking and embedding pipelines. For option 1, we would need to own that complexity ourselves.

Important constraints

• We do not require real-time processing. Near-real-time or batch is perfectly acceptable.
• Volume is modest — approximately 1,000 documents per day, possibly fewer.
• A key functional requirement that appears to be driving the Cosmos DB proposal is the need for certain users to visually compare the original physical document against its JSON representation, validate the output, and make corrections directly if needed. Cosmos DB is being considered as the interface for that editorial workflow. if i undertood it correbtly from what was told to me ...

Specific questions

1. When do we actually need Cosmos DB  ? Which use cases genuinely cannot be served by simply exposing the Silver Delta tables as a JSON document store which would be composed by two tables (metadata of the file plus content table, both linked via document_id)?
2. If we use Cosmos DB and exclude Azure AI Services, should the chunks produced in Silver also be stored there? And where should embeddings be generated — in Silver, or natively within Cosmos DB during the write process?
3. Would you recommend using both Cosmos DB and Azure AI Search together? If so, should they be connected to each other, or should Azure AI Search index directly from Silver while Cosmos DB sits in Gold serving only the original documents (without chunks or embeddings)?
4. Should end users be permitted to edit payloads directly in Cosmos DB, or should that interaction be governed and controlled somehow differently?

Any guidance on how to think through this architecture would be greatly appreciated.

Many thanks, Pedro