general question
35 TopicsSQL Database capacity consumption too high !
I'm copying a few massive tables from an On-prem SQL Server Database into a Fabric SQL Database. Tables are around 1.6TB, 1.3TB and 1.1TB, and there are more smaller tables as well. The Database has reached maximum allocated size and I've stopped copying two days ago. One of the batches abruplty stopped during load. It is still consuming a LOT of CUs ! I'm aware of how even after the copy data activity has stopped there's prolonged capacity consumption (apparently for indexing, statistics etc) but this is a slightly higher percentage of base capacity than what I've seen before and its been going on for much longer than before. Could this be because of the size of the item (~3416 GB) ? Also, is there any way to suppress this without having to wait until it dies out, whenever that it. I cannot apply the per-database max vCore setting (as proposed in this solution) because of the size.14Views0likes1CommentSQL database in Fabric: how fresh is the OneLake replica, and what doesn't get mirrored?
Hi all, I'm considering SQL database in Fabric as the operational (OLTP) backend for an internal app, and using its automatic replication to OneLake for analytics instead of building a separate ingestion pipeline. My understanding is that data written to the SQL database is automatically replicated to OneLake in Delta format and can be queried through the SQL analytics endpoint, Spark, or Power BI. Before relying on that, I'd like to understand the details. Questions: Freshness: what replication latency do people see in practice between a committed transaction and the data appearing in OneLake? Does it change under heavy write load? Coverage: which tables, columns or data types are not replicated (for example, certain data types, tables without primary keys, or computed columns)? Is there a way to check replication status for each table? Schema changes: what happens to the OneLake copy when I add, drop or alter a column, or rename a table? Does replication continue, or does the table need to be reseeded? Consistency: if a single transaction updates several tables, do they show up in OneLake together, or can analytics queries briefly see a partial state? Capacity cost: does replication use capacity units separately from the database workload, and is there a way to monitor it? Design: for near-real-time reporting, would you query the replica directly, or still build curated tables in a Warehouse or Lakehouse on top of it? Thanks!Solved35Views0likes3CommentsHow can I convert JSON data to SQL format without writing a custom script?
I have a large JSON dataset that I need to migrate into a SQL database for further analysis and reporting. The JSON contains nested objects and multiple records, so manually creating SQL statements is becoming time-consuming. I’m looking for a reliable way to convert JSON to SQL while keeping the data organized and minimizing manual work. I came across the SysTools JSON Converter, which supports JSON to SQL conversion and seems useful for handling such requirements. A few things I’m particularly looking for: Support for large JSON files and bulk conversion Ability to handle nested JSON data Preserve the available data hierarchy and structure Export JSON directly into SQL format An easy-to-use interface without requiring extensive coding Offline processing for better control over sensitive data Has anyone used SysTools JSON Converter Tool for JSON-to-SQL conversion? I’d appreciate feedback on its performance and output quality.Solved70Views0likes3CommentsSQL Strategy
Hi everyone, With the growing number of database technologies available, I'm curious how organizations decide which database should support different workloads. When working with Microsoft Fabric, how do you decide whether data should remain in an operational database or be moved into Fabric for analytics? I'd appreciate hearing the decision-making process you've found most effective.Solved240Views0likes3CommentsTraining course recommendation on learning Natural Language to SQL within SQL Server?
Hi (I hope this is a suitable forum), We currently get our insights through Power BI (PBI) accessing data within SQL Server. Producing thsese reports/insights is labour intensive. With the introduction of AI, I've been tasked to find our if we are able to write questions (in Natural Language) of our data (held within SQL Server) and have the answers returned. If this is possible then it may negate the need for PBI reports to be generated to achieve the same insights. I have no understanding of AI at present. My gut is telling me the data model will have to be specific and very descriptive, and then we'd have to rely on customers (end-users) asking their questions in a regimented way - all challenges. I've been reading various online articles on the subject but each article, it seems, recommends different software and I'm gaining no momentum. Therefore, I am here to ask if you can advise on any training courses that would take me from a complete novice to someone who can, either, get the aim working or is equipped to advise it wouldn't work with our setup. We are a Microsoft (MS) house (SQL Server, Dynamics, etc...) and all in Azure. Any training material should follow using MS technologies otherwise the I.T. dept may put a blocker on it. Your suggestions will be most welcome. Thanks.Solved1.6KViews3likes6CommentsForeign keys not visible in database explorer
Hi, when I create a relationship between two tables I've noticed that the related foreign key isn't visible in the database explorer, while for the primary keys at least it is reported the "pk" acronym: this is it a limitation or a regular behaviour? ThanksSolved1.8KViews0likes8CommentsDifferent behaviour between SQL database and SQL Analytics endpoint
Hi, I've noticed that there are diffencies between SQL database view and SQL Analytics endpoint view in the explorer. F.e. when I create a table with a datetime column I can do it successfully in the SQL database view while in the SQL Analytics endpoint view I need to use a datetime2 column. Moreover, it could occur that a dropped table in the SQL database view remains in the SQL Analytics endpoint view. The SQL Analytics endpoint view should correspond to SQL database view. ThanksSolved2.4KViews0likes12CommentsGeneric error during SQL database creation
Hi, in a Fabric workspace with a F32 capacity (in North Europe region) I'm trying to create a SQL database for the first time, but I've only a generic error that says "A problem is occured". This message doesn't help to understand the real issue. I'm an admin for the workspace. Any suggests to me, please? ThanksSolved1.2KViews0likes1CommentFabric SQL Database GA
Hi all, Any idea when Fabric SQL Database will switch to GA? I saw an earlier post saying probably Q3 and definitely by the end of the year, but not seen anything since. Calendar Q3 has gone and I can't see a confirmed data in the roadmap. Thanks, DBSolved6.7KViews1like2Comments