Passing a bearer token to copy data action using GET
Hi All, Not a question, but something I ran into, as I am just picking up MS Fabric, pipelines etc. I needed to use web request, to get an access token that is valid for 30 minutes. Next I needed to use that token as a bearer token to do a follow up API call to get actual data , there didn't seem to be that much info around, when I tried to google it. Eventually I found a you tube video, that told me what i needed to know. (linked below). Things like apikeys and other credentials will of course vary on the API yo are trying to use. This is my flow. Step 1 Do your web request Step 2 link the Web and copy data action Step 3 In the copy data action set it up as follows: As you can see the value of the Authorization header is dynamic, with in my case, the following expression: Bearer @{activity('Get access token').output.access_token} The part ".access_token" depends on the json you get returned, in my case it was a very flat structure with no above array. { "token_type": "BearerToken", "issued_at": "xxx", "salesforce_id": "xxx", "access_token": "xxx", "application_name": "xxx", "scope": "refresh_token full", "expires_in": "xxx" } Above you can see where i got the parameter access_token from. If this array had been nested in for instance another array called "data" then the value would have been data.access_token in stead of just access_token. I hope this is usefull info to share here. The video I got this from you can find here . Cheers Hans
Materialized Lake View Refresh activity Pipeline bug
Hi all, When using the preview feature refresh mlv activity in the pipeline, is do have some major bugs. After saving, it goes back to default, the pipeline does not want to refresh after, and I cannot parametrize it. Is there any workarround, and is this a known bug? Cause this is not workable.
Error - Using the MongoDB Om-Premise Connector
Hello everyone, I'm trying to establish a connection between Fabric and an on-premises MongoDB server. https://learn.microsoft.com/en-us/fabric/data-factory/connector-mongodb-overview In fact, the scenario involves writing data from Fabric to an on-premises MongoDB NoSQL database. The connection has been successfully established on the gateway side (with all necessary firewall rules configured). Then I see the connection in the “Copy Data” section of the “Data Pipeline.” So that means the mapping is working properly. However, the connection test isn't working... Of course, the error message doesn't help at all. For your information, here is the server pattern specified in the gateway connection mongodb01.example.com:27017,mongodb22.example.com:27017,mongodb03.example.com:27017/?authMechanism=DEFAULT&authSource=xxxxxxxxx Thank in advance for your help Have a nice day, Vivien
Dynamically set source connection in copy activity and connection in lookup, script activity
Wanted to understand how we can provide the source connection dynamically rather than manually selecting from the dropdown for copy activity, lookup, script, get metadata activity, etc. as we have to deploy the pipeline to other workspaces which uses different connection name (DEV/QA/UAT/PROD).
Fabric copy activity fails when JSON source is missing attributes
I have created a Fabric Copy activity that gets data from a REST API connection. I was able to apply the "Collection Reference" as $['results'] node of the JSON response and am writing the inner structs to different fields of a table. The copy activity fails when one of the structs is missing from any API response, and I would like to define a strict mapping that uses NULL or default value for any struct or attribute that is missing from the source instead of failing with an error. Can you please help with a better solution to improve the fault tolerance here?
Fabric Copy Activity from S3 issue: Could not establish trust relationship for the SSL/TLS channel
Hello, I'm trying to copy data from an S3 bucket into OneLake using Copy data activity in a data pipeline in Fabric. While configuring the source for the Copy Data activity, I'm using connection type Amazon S3, Connection is established successfully while clicking Test Connection but when trying to browse files from root, nothing gets listed (empty screen). When I enter the bucket name in the File path and choose "Browse from specified path" option, the following error is raised: "The file operation is failed. A WebException with status TrustFailure was thrown. The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. The remote certificate is invalid according to the validation procedure. Activity ID: xxxxx " Any help?? Thank you!
