Forum Discussion
Error Retrieving data from dataverse with a pipeline
hey everyone burakkaragoz tayloramy Pragati11 ,
thank you for everyone who assisted, just wanted to let you in on what was the issue and that it was fixed thankfully.
so i opened a ticket with microsoft to check on everything and get help, but, as you guys have mentioned, microsoft did as well, and their conclusion was to eventually split the data by day, since i have a large amount of monthly data (over 15m a month), which i had started doing on friday night, until saturday noon, i was going through the dataverse environment, i wanted to check the table's logical name, and by mistake went to the "connections" tab. there, i found the email we're using as the admin on the fabric workspace, and the status was "disconnected", so i re connected it, re ran the pipeline, ran for 24hours straight and pulled 30 million rows! that way i can make only around 4 pipelines to pull all data instead of doing it daily for each month.
This looks like a Firewall is blocking the connection.
Have you tried Workspace Identity approach?
https://learn.microsoft.com/en-us/fabric/security/workspace-identity
Hi Pragati11 ,
thank you for the reply!
as i have mentioned above, i am now trying the run with the workspace identity set on my dataverse connection instead of the org account.
i will get back with feedback once its hopefully resolved.
thank you!
Unfortunately, that didnt work either
This then looks more firewall blocking issue to me which you need to get sorted on the Dataverse side.
Hi Pragati11 , okay i'll have to check that with the dataverse team
Hi Hussein_charif,
I agree with Pragati11, this looks like a firewall issue, not an authenciation isssue.
If you found this helpful, consider giving some Kudos. If I answered your question or solved your problem, mark this post as the solution.
