Forum Discussion

marcoG's avatar
marcoG
Advocate III
2 years ago
Solved

Why does SQL Server return some rows during query execution but Lakehouse in Fabric doesn't?

In general there are some operators that require all rows to come to them before they can start processing them and pass them downstream, there are other operators that are able to start streaming or...
data engineering
Data Warehouse
  • marcoG's avatar
    marcoG
    2 years ago

    Hi Anonymous ,

    I don't think it's a problem with my workspace, I wish it were.
    It is clearly a general problem as the performance of the Azure Synapse Serveless Pools is superior to that of the Fabric SQL endpoints and furthermore the Azure Synapse serveless pools guarantee, like the other databases, the incremental fetch of the response while in the Fabric SQL endpoint this does not happen by having to wait for the query to complete before getting the response.
    Thanks anyway for the attempt to support
    Greetings

marcoG's avatar
marcoG
Advocate III
2 years ago

Hi AndyDDC ,

HI
I have the impression that the Fabric lakehouse and data warehouse SQL endpoints perform less well than the Azure Synapse serveless SQL pools.
How to test I used an F64 capacity but while in Fabric I have to wait several minutes to get a Power BI paginated report rendered (without the possibility of seeing the progressive loading of the rows) in Azure Synapse Serveless Pool this happens within the minute with a management of loading incremental rows

AndyDDC's avatar
AndyDDC
Most Valuable Professional
2 years ago

Hi marcoG I'm suprised you're seeing worse performance in Fabric vs Serverless SQL Pools.  What is your underlying data in Serverless? parquet/delta?  If so are the file sizes the same (eg compaction and vacuuming happening).  What's the performance like if you run the SQL query that the paginated report generates on the sql endpoint vs serverless?

  • marcoG's avatar
    marcoG
    Advocate III
    2 years ago

    For the moment it's just a feeling, we would need some comparative tests of the 2 products.

    I haven't found anything to this effect online.
    In any case, on serveless pools I use external tables of parquet files on adls gen2 while on fabric I obviously use delta tables but the underlying data is the same.
    In my opinion, the access latency to Onelake Vs Adls Gen2 and above all the execution plans Fabric SQL Vs Azure Serveless Pool should be better investigated