data integration

6 Topics

Authentication options for SharePoint Folder in Data Flow Gen2
Hi Experts, I come to you with a question that is bothering me. Well, in my project I have a Data Pipeline that uses Data Flow Gen 2 which has a connection to a SharePoint folder and pulls data from several excel files. Authentication is done via an account. The problem is that for this account the password is renewed from time to time and the connection can go down. Hence my question, are there any other methods to connect to SharePoint Folder in a better and less error-prone way? I was thinking about Service Principal but I have read that it gets deactivated every 100 days. Should I use another connection for ‘Web’ data or something like that to have a connection not based on the account I am using? Any suggestions would be greatly appreciated. Best regards, Konrad
Solved
Konrad909090
1 year ago Place Dataflow
3.1KViews
0likes
6Comments
Parametarized relative URL in Microsoft Fabric
Hi, We are very new to Microsoft Fabric. I am trying to create copy activity using REST API linked service. We have to connect to API URL and run daily job supplying Yesterdays date and other parameter values. Manually we were able to pull data (adding parameter values in relative URL) but we need to pass the date values dynamically so we can schedule the job. Example base URL : https//:api.indigo.com Relative URL : /z1/ktdata/sale/data/?expd=FALSE&pdt='yesterday date'&offset=0&lmt=10000 where we wanted to pass expd, pdt, lmt values dynamically; especially the pdt which is previous day date in yyyy-mm-dd format. Any help appreciated.
Solved
krishn
2 years ago Place Pipelines
4KViews
0likes
4Comments
Dataflow Gen2 fast copy with On-premises gateway
i noticed recent update https://blog.fabric.microsoft.com/en-us/blog/boost-performance-and-save-costs-with-fast-copy-in-dataflows-gen2?ft=All and i have tested with the 8M rows table but it seems it still does not support fast copy with On-premises gateway. any idea when would it be supported?
Solved
jwryu
2 years ago Place Dataflow
1.7KViews
0likes
3Comments
Microsoft Fabric Data Pipeline + Power BI API Scanner
Hello, I'm looking to set up a data pipeline Fabric, I started by creating a copy data activity to retrieve a list of workspaces. But then I'm stuck, I can't manage to break down this list into packages of 100. Then for each block of 100 call the PostWorkspaceInfo API. Does anyone have any ideas? Best regards, Charline
Charline_74
2 years ago Place Pipelines
2.1KViews
0likes
5Comments
What is the difference between external data and an integration dataset in Azure Synapse?
What is the difference between external data and an integration dataset in Azure Synapse?
HamidBee
3 years ago Place Pipelines
1.6KViews
0likes
3Comments
A typo in the Data pipeline (in Korean)
Hi, Team I was trying to set up a Copy data task in the Data pipeline. That should be changed as shown below. 소스 -> 원본 대상 주소 -> 대상 Korean English Thanks, Hong
Solved
Hongju_Jung
3 years ago Place Pipelines
793Views
0likes
1Comment