Forum Discussion

Nalapriya's avatar
Nalapriya
New Member
1 year ago
Solved

KQL Database Benchmarks

I am looking for benchmark statistics on the volume, range of data, and performance capabilities of KQL for handling high-velocity data. Specifically, I would like to know the numeric values related to the data volume KQL can process efficiently, including throughput, latency, and any scalability metrics.

  • Latency, ahahaha.  Expect a minimum latency of 8 minutes before the inserted data is retrievable.

  • Anonymous's avatar
    Anonymous
    1 year ago

    Hi Nalapriya ,

    Thanks for lbendlin's reply!
    And Nalapriya , There may not be any document that specifically describes the values ​​of these attributes.

    Generally speaking:
    Data Ingestion Rates: Azure Data Explorer can ingest data at rates of up to 200 MB/sec per node. In large clusters, this can scale linearly, allowing ingestion rates in the range of GBs per second.
    Data Storage: ADX can handle petabytes of data efficiently.
    Latency: Query latency is typically in the range of milliseconds to seconds, depending on the complexity of the query and the volume of data being scanned. Simple queries on indexed data can return results in milliseconds.
    Horizontal Scaling: ADX clusters can scale out by adding more nodes, allowing for linear scaling of both ingestion and query performance. A single cluster can scale to hundreds of nodes.

    I have found several relevant official documents for you, hoping that they will be helpful to you:
    Query limits - Kusto | Microsoft Learn
    Eventhouse and KQL Database consumption - Microsoft Fabric | Microsoft Learn

    Here are some commands to set these properties:
    Request limits policy - Kusto | Microsoft Learn
    Also these ones:


    I also found a blog that has some guidance on how to do your own KQL-related performance testing, which I hope will be helpful to you:
    Evaluate query performance of Azure Data Explorer | Microsoft Community Hub

    Best Regards,
    Dino Tao
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

2 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi Nalapriya ,

    Thanks for lbendlin's reply!
    And Nalapriya , There may not be any document that specifically describes the values ​​of these attributes.

    Generally speaking:
    Data Ingestion Rates: Azure Data Explorer can ingest data at rates of up to 200 MB/sec per node. In large clusters, this can scale linearly, allowing ingestion rates in the range of GBs per second.
    Data Storage: ADX can handle petabytes of data efficiently.
    Latency: Query latency is typically in the range of milliseconds to seconds, depending on the complexity of the query and the volume of data being scanned. Simple queries on indexed data can return results in milliseconds.
    Horizontal Scaling: ADX clusters can scale out by adding more nodes, allowing for linear scaling of both ingestion and query performance. A single cluster can scale to hundreds of nodes.

    I have found several relevant official documents for you, hoping that they will be helpful to you:
    Query limits - Kusto | Microsoft Learn
    Eventhouse and KQL Database consumption - Microsoft Fabric | Microsoft Learn

    Here are some commands to set these properties:
    Request limits policy - Kusto | Microsoft Learn
    Also these ones:


    I also found a blog that has some guidance on how to do your own KQL-related performance testing, which I hope will be helpful to you:
    Evaluate query performance of Azure Data Explorer | Microsoft Community Hub

    Best Regards,
    Dino Tao
    If this post helps, then please consider Accept it as the solution to help the other members find it more quickly.

  • Latency, ahahaha.  Expect a minimum latency of 8 minutes before the inserted data is retrievable.