Forum Discussion

Nalapriya's avatar
Nalapriya
New Member
1 year ago
Solved

Ingest data from Azure Event hub using Event stream

There are static json files in Azure Event hub and I'm trying to ingest it into KQL Database using Event stream. I'm able to connect to event hub using event stream but I'm not able to preview the da...
  • nilendraFabric's avatar
    1 year ago

    Hi Nalapriya 


    This is the default behaviour of eventhub.

     

    The issue you’re experiencing with ingesting static JSON files from Azure Event Hub into a KQL Database using Event Stream is likely related to the nature of the data in your Event Hub. Event Stream is designed to work with real-time, streaming data, and may have difficulty processing static files that are not actively being pushed into the Event Hub.

     

    Event Hub is designed for real-time event streaming. If your JSON files are static and not being actively sent to the Event Hub, Event Stream may not be able to detect or process them.

     

    Event Hub has a limited data retention period. If the static files were added to the Event Hub some time ago, they may have expired and been removed.


    Potential solution:

     

    Simulate Real-time Data: Instead of using static files, try simulating real-time data by periodically sending the JSON content to your Event Hub. This can be done using a script or an Azure Function that reads the static files and sends them as events.

     

     

    For static data, consider using other ingestion methods supported by KQL Database, such as direct ingestion from blob storage or using Azure Data Factory or Fabric pipelines.

     

    Use Event Hub Data Generator: If you’re just testing the setup, you can use the Event Hub Data Generator feature in the Azure Portal to send sample data to your Event Hub and verify the Event Stream connection.

     

     

    If this is helpful please accept the solution and give kudos.

     

    thanks