Forum Discussion
Relational database as a destination in Fabric
Why not just use a Warehouse table, Lakehouse table or even a csv or json in Lakehouse files?
Why can't you use a Delta Tables on the Lakehouse? They are ACID-compliant and relational. Just because they are capable of handling large datasets does not mean they are incapable of handling small datasets.
If you are just reading meta and configurations, you don't even need a relational database. You can just write that data to a file (XML, JSON, CSV, etc.) in a Lakehouse and read that file whenever you need it.
If it's smaller than 5mb, you could even store it in a variable in a data pipeline and retrieve it in real time when you need to refresh it.
Why can't you use a Delta Tables on the Lakehouse? They are ACID-compliant and relational. Just because they are capable of handling large datasets does not mean they are incapable of handling small datasets.
If you are just reading meta and configurations, you don't even need a relational database. You can just write that data to a file (XML, JSON, CSV, etc.) in a Lakehouse and read that file whenever you need it.
If it's smaller than 5mb, you could even store it in a variable in a data pipeline and retrieve it in real time when you need to refresh it.