Migrating Azure Function with Queue Storage Trigger to Microsoft Fabric for ETL Processing

[yacevedo]

Hello,

I'm currently facing a challenge in my project and looking for advice or best practices on how to transition to a new setup.

**Current Setup:**
I have an Azure Function that is triggered by Queue Storage, which is continuously listening for incoming messages. Upon receiving a message, the function reads it and then asynchronously calls another function to process the message and perform ETL (Extract, Transform, Load) operations. This setup ensures that messages are processed in real-time, as they arrive.

**Goal:**
My objective is to migrate this continuously listening and processing scenario to Microsoft Fabric. I'm interested in understanding the best way to implement this to maintain or even enhance the efficiency of real-time message processing, while leveraging the scalability, management, and orchestration capabilities that Fabric offers.

**Specific Questions:**
1. How can I achieve a continuous listening mechanism for incoming messages within Fabric, similar to the Queue Storage trigger in Azure Functions?
2. What are the best practices for implementing asynchronous and real-time message processing in Fabric?
3. Are there any recommended patterns or services in Fabric specifically suited for real-time ETL operations?
4. How does Fabric facilitate the orchestration of these microservices, particularly in handling errors and implementing retry logic for uninterrupted processing?

I would greatly appreciate any insights, documentation links, or examples of similar setups. Thank you for your support and guidance.

Best regards,
Yoel

[Anonymous]

Hello @yacevedo ,

Thanks for using Fabric Community.
At this time, we are reaching out to the internal team to get some help on this .
We will update you once we hear back from them.

[Anonymous]

Hi @yacevedo ,

Apologies for the delay in reply from our side. 
Unfortunately we didn't get any update from the internal team regarding the ask.
I kindly request you to raise a support-ticket here. Our team will coordinate with you and help you with the same.

After creating a Support ticket please provide the ticket number as it would help us to track for more information. Hope this helps. Please let us know if you have any other queries.

[Anonymous]

Hello @yacevedo ,

We haven’t heard from you on the last response and was just checking back to see if you got a chance to create a support ticket?
In case if support ticket is created, please provide the ticket number as it would help us to track for more information.