Forum Discussion

Anonymous's avatar
Anonymous
Not applicable
1 year ago
Solved

How to ingest the data directly into Eventhouse's DB

I tried to ingest the data into Fabric Event House KQL DB.

First, I get the KUSTO Access token using this command

curl --location 'https://login.microsoftonline.com/<tenant-id>/oauth2/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=client_credentials' \
--data-urlencode 'client_id=<client-id>' \
--data-urlencode 'client_secret=<client-secret>' \
--data-urlencode 'resource=https://api.kusto.windows.net'​


Then I used this token to list the event houses [Working]

curl --location --request GET 'https://api.fabric.microsoft.com/v1/workspaces/<workspace-id>/eventhouses/<event-house-id>' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '{
    "csl":".get ingestion resources"
}'

This is the sample response, i received
{
    "id": "XXXXX-XXXX-XXXX-XXXX-f6fa97c409e7",
    "type": "Eventhouse",
    "displayName": "test-eventhouse",
    "description": "test-eventhouse",
    "workspaceId": "XXXXXX-XXX-XXXX-XXXX-fa3c5ebfb244",
    "properties": {
        "queryServiceUri": "https://trd-XXXXXXXXXX.z0.kusto.fabric.microsoft.com",
        "ingestionServiceUri": "https://ingest-trd-XXXXXXXXXX.z0.kusto.fabric.microsoft.com",
        "databasesItemIds": [
            "XXXXXXX-XXXX-XXXX-XXXX-bc61a54af016"
        ],
        "minimumConsumptionUnits": 4.25
    }
}


Then tried to using the ingestion-uri returned from the above response to ingest data

curl --location 'https://ingest-trd-XXXXXXXXXX.z0.kusto.fabric.microsoft.com/v1/rest/ingest/test-eventhouse/products?streamFormat=JSON&mappingName=mymapping' \
--header 'Content-Type: application/json' \
--header 'Authorization: ••••••' \
--data '[
  { "product": "Laptop", "price": 75000 },
  { "product": "Smartphone", "price": 30000 },
  { "product": "Tablet", "price": 20000 }
]'


Result of above API: [Failure]

HTTP Status: 404
Not found: There is no resource at this URI path: '/v1/rest/ingest/test-eventhouse/products'
Error details:
ClientRequestId='unspecified;XXXXX-XXXX-XXXX-bb6f-b69ba8855def', ActivityId='13411632-2b6d-498d-92b1-8d3d1a0b0afd', Timestamp='2025-09-04T07:46:16.4756650Z'.

​



So, i replaced the ingestion URI with this

https://help.kusto.windows.net/v1/rest/ingest/test-eventhouse/products?streamFormat=JSON

But it returns this response

HTTP Status: 404
NotFound_StreamingIngestionDisabledForCluster: Streaming ingestion endpoint is disabled. Error details: ClientRequestId='unspecified;XXXX-XXXX-XXXX-947f-87bf31fb19ac', ActivityId='b80b0990-7943-4bf3-b2dd-c34994d39572', Timestamp='2025-09-04T11:36:43.8157598Z'.


I'm not sure whether I can directly ingest the data into EventHouses's DB via REST API? Please help me

Additional Information:
I've enabled stream ingestion policy for my database, I have already created a DB products with columns [product, price].


Available Databases

 

{"Tables":[{"TableName":"Table_0","Columns":[{"ColumnName":"TableName","DataType":"String","ColumnType":"string"},{"ColumnName":"DatabaseName","DataType":"String","ColumnType":"string"},{"ColumnName":"Folder","DataType":"String","ColumnType":"string"},{"ColumnName":"DocString","DataType":"String","ColumnType":"string"}],"Rows":[["products","XXXXX-XXXX-XXXXX-XXXX-bc61a54af016",null,null],["bicycles","XXXXX-XXXX-XXXX-XXXX-bc61a54af016",null,null]]}]}​

 


Table Schema Returned Via API

{"Tables":[{"TableName":"Table_0","Columns":[{"ColumnName":"AttributeName","DataType":"String"},{"ColumnName":"AttributeType","DataType":"String"},{"ColumnName":"ExtentSize","DataType":"Int64"},{"ColumnName":"CompressionRatio","DataType":"Double"},{"ColumnName":"IndexSize","DataType":"Int64"},{"ColumnName":"IndexSizePercent","DataType":"Double"},{"ColumnName":"OriginalSize","DataType":"Int64"},{"ColumnName":"AttributeId","DataType":"Guid"},{"ColumnName":"SharedIndexSize","DataType":"Int64"},{"ColumnName":"StorageEngineVersion","DataType":"String"},{"ColumnName":"SamplePercent","DataType":"Double"},{"ColumnName":"IncludeColdData","DataType":"Boolean"}],"Rows":[["product","StringBuffer",0,0.0,0,0.0,0,"XXXXX-XXXX-XXXX-b318-3293a32083e5",0,null,100.0,false],["price","Decimal",0,0.0,0,0.0,0,"XXXXX-XXXX-XXXX-XXXX-e5ec071ecdc7",0,null,100.0,false]]}]}


This is the response of stream ingestion policy [Used KQL editor]

KQL
.show database ['test-eventhouse'] policy streamingingestion;


Response
"PolicyName": StreamingIngestionPolicy,
"EntityName": [XXXX-XXXX-XXXX-XXXX-bc61a54af016],
"Policy": {
  "IsEnabled": true,
  "HintAllocatedRate": "2.100"
},
"ChildEntities": [
  "products",
  "bicycles"
],
"EntityType": Database


Referred Docs:
https://learn.microsoft.com/en-us/kusto/api/rest/streaming-ingest?view=microsoft-fabric
https://learn.microsoft.com/en-us/kusto/management/streaming-ingestion-policy?view=azure-data-explorer

 

4 Replies

  • v-lgarikapat's avatar
    v-lgarikapat
    Icon for Community Support rankCommunity Support

    Hi Anonymous ,

    Thanks for reaching out to the Microsoft fabric community forum.

    lbendlin 

    Thanks for your prompt response.

    Anonymous 

    I wanted to follow up and confirm whether you’ve had the opportunity to review the information  provided by lbendlin  If you have any questions or need further clarification, please don’t hesitate to reach out.

     

    We appreciate your collaboration and support!

    Best regards,
    Lakshmi.

    • v-lgarikapat's avatar
      v-lgarikapat
      Icon for Community Support rankCommunity Support

      Hi Anonymous ,

       

      We’d like to confirm whether your issue has been successfully resolved. If you still have any questions or need further assistance, please don’t hesitate to reach out. We’re more than happy to continue supporting you.

      We appreciate your engagement and thank you for being an active part of the community.


      Best Regards,
      Lakshmi.

      • v-lgarikapat's avatar
        v-lgarikapat
        Icon for Community Support rankCommunity Support

        Hi Anonymous ,We’d like to confirm whether your issue has been successfully resolved. If you still have any questions or need further assistance, please don’t hesitate to reach out. We’re more than happy to continue supporting you.

        We appreciate your engagement and thank you for being an active part of the community.


        Best Regards,
        Lakshmi.