Forum Discussion

Asam3833's avatar
Asam3833
New Member
5 months ago
Solved

Data replication from AKS

Hi , When workloads run on K8s, what’s the cleanest way to land data into OneLake (events vs files vs APIs) ?
  • deborshi_nag's avatar
    5 months ago

    Hello Asam3833 

     

    The best approach for moving your data to Fabric depends on the type of data you have.

     

    If your data includes things like system activity, logs, or real-time information from services or platforms such as Kafka, Event Hubs, or IoT Hub, then choose Events. This is ideal when you want to analyse data as it arrives, receive alerts, or quickly send information to tools like KQL, Lakehouse, Notebooks, or Power BI. You can send data from your applications using your current systems, set up simple data flows without coding, and make your data available in OneLake by enabling the right settings.

     

    If your data is already organised in batches, such as Parquet, CSV, or JSON files, choose Files. This works well for jobs that process data in groups, providing a cost-effective and efficient way to move large amounts of information. You can upload files directly to OneLake using familiar methods, and use standard authentication. This is commonly used in automated data pipelines.

     

    If you need to manage your data storage or trigger updates, use the Fabric APIs. This is suitable for tasks like creating or updating Lakehouses, or running table updates, but not for adding individual rows. If your systems are already saving files in cloud storage, you can make them available in Fabric immediately using OneLake Shortcuts, without extra steps. For private storage, use Trusted Workspace Access.