Error connecting with Amazon Kinesis Stream in Fabric using Eventstream

[Dataseeker009]

Hello friends, I have problem which I'm trying to debug for last few days. I hope somebody can give an approach or solution to this issue.

I have AWS kinesis stream which pushes data every 15 to 30 seconds. I want to create a power BI report. I created an event stream in Fabric, and used a Amazon kinesis stream connector and I have the sink location to EventHouse. Now the issue is Amazon kinesis connector is throwing an error while validating the connection. I have the error as "connector xxxxxxx failed, Reason: E0050002 Kinesis connection validation failed. Please check your kinesis configuration. Details: Org.ache.kafka.config.configException: Failed to validate  connection to kinesis stream xxxxx in region us-west-2"

I checked the connection details on AWS kinesis stream like name of stream, region, key id and key secrets everything is same, I have no firewall or virtual network on top AWS layer which would prevent any connection from validation but still the connector doesn't work, error looks genric.

Any help on this issue would help me a lot.

[v-tsaipranay · Accepted solution]

Hi @Dataseeker009 ,

Thanks for reaching out to the Microsoft fabric community forum

We understand you are experiencing issues connecting your AWS Kinesis stream to Microsoft Fabric for Power BI reporting. The error message "E0050002 Kinesis connection validation failed. Org.apache.kafka.config.ConfigException" suggests there might be an issue with the connection parameters or permissions.

Here are some steps we recommend helping you troubleshoot and resolve this issue:

• Stream name (case-sensitive!), region (us-west-2), and AWS credentials in Fabric. Try new AWS keys.
• Ensure your IAM role has kinesis:DescribeStream, kinesis:GetRecords, kinesis:GetShardIterator, and kinesis:ListStreams.

• Please run the command: aws kinesis describe-stream --stream-name --region us-west-2. If this command fails, the issue is likely with AWS, not Fabric.

• Please verify if any firewalls are blocking outbound connections.

•  If all other options have been exhausted, please delete and recreate the Fabric event stream and connection.

For detail information for adding Amazon Kinesis Data Streams source to an eventstream, please refer the follwing documentation link:

Add Amazon Kinesis Data Streams source to an eventstream - Microsoft Fabric | Microsoft Learn

I hope my suggestions give you good ideas, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

[v-tsaipranay · Accepted solution]

Hi @Dataseeker009 ,

Thanks for reaching out to the Microsoft fabric community forum

We understand you are experiencing issues connecting your AWS Kinesis stream to Microsoft Fabric for Power BI reporting. The error message "E0050002 Kinesis connection validation failed. Org.apache.kafka.config.ConfigException" suggests there might be an issue with the connection parameters or permissions.

Here are some steps we recommend helping you troubleshoot and resolve this issue:

• Stream name (case-sensitive!), region (us-west-2), and AWS credentials in Fabric. Try new AWS keys.
• Ensure your IAM role has kinesis:DescribeStream, kinesis:GetRecords, kinesis:GetShardIterator, and kinesis:ListStreams.

• Please run the command: aws kinesis describe-stream --stream-name --region us-west-2. If this command fails, the issue is likely with AWS, not Fabric.

• Please verify if any firewalls are blocking outbound connections.

•  If all other options have been exhausted, please delete and recreate the Fabric event stream and connection.

For detail information for adding Amazon Kinesis Data Streams source to an eventstream, please refer the follwing documentation link:

Add Amazon Kinesis Data Streams source to an eventstream - Microsoft Fabric | Microsoft Learn

I hope my suggestions give you good ideas, if you need any further assistance, feel free to reach out.

If this post helps, then please give us Kudos and consider Accept it as a solution to help the other members find it more quickly.

Thank you.

[Dataseeker009]

thank you @v-tsaipranay the source stream from AWS was behind firewall and wasn't accepting the connection. Once it was public we were able to access it. thanks

[v-tsaipranay · Accepted solution]

Hi @Dataseeker009 ,

I wanted to check if you had the opportunity to review the information provided. Please feel free to contact us if you have any further questions. If my response has addressed your query, please accept it as a solution and give a 'Kudos' so other members can easily find it.

Thank you.

[v-tsaipranay]

Hello @Dataseeker009 ,

I wanted to follow up on our previous suggestions regarding Error connecting with Amazon Kinesis Stream in Fabric using Eventstream. We would love to hear back from you to ensure we can assist you further.

If my response has addressed your query, please accept it as a solution and give a ‘Kudos’ so other members can easily find it. Please let us know if there’s anything else we can do to help.

Thank you.

[v-tsaipranay]

Hello @Dataseeker009 ,

May I ask if you have resolved this issue? If so, please mark the helpful reply and accept it as the solution. This will be helpful for other community members who have similar problems to solve it faster.

Thank you.