Forum Discussion
Initial load of spark streams
- Anonymous1 year ago
Hi innocence84
Thank you for reaching out microsoft fabric community forum.You have two tables:
- SALES: one-time historical load from SQL Server
- SALES_delta: ongoing new transactions via streaming from Kafka
To get a complete view of sales, you should combine them into a single table (e.g., SALES_silver) after the historical load is done and streaming is stable.
How to combine:
- If SALES_delta only has new rows, use INSERT to append to SALES_silver.
- If it includes updates or late data, use a Delta MERGE to handle both inserts and updates.
This unified table (SALES_silver) becomes your single source of truth, and you can stop querying SALES and SALES_delta separately.
If this solution helps, please consider giving us Kudos and accepting it as the solution so that it may assist other members in the community
Thank you.
Hi innocence84
I hope this information is helpful. Please let me know if you have any further questions or if you'd like to discuss this further. If this answers your question, please Accept it as a solution and give it a 'Kudos' so others can find it easily.
Thank you.