data engineering
7 TopicsParametarized relative URL in Microsoft Fabric
Hi, We are very new to Microsoft Fabric. I am trying to create copy activity using REST API linked service. We have to connect to API URL and run daily job supplying Yesterdays date and other parameter values. Manually we were able to pull data (adding parameter values in relative URL) but we need to pass the date values dynamically so we can schedule the job. Example base URL : https//:api.indigo.com Relative URL : /z1/ktdata/sale/data/?expd=FALSE&pdt='yesterday date'&offset=0&lmt=10000 where we wanted to pass expd, pdt, lmt values dynamically; especially the pdt which is previous day date in yyyy-mm-dd format. Any help appreciated.Solved4KViews0likes4CommentsQUESTION::PIPELINE::SET VARIABLE::RACE CONDITION
Let's say you have a situation like this one: and there is only one variable, meaning the variable being referenced by both Set variable activities is the same. And after the Set variable, to the right (not shown here), the next activity, say a IF condition, depends on the value of that one variable. Can the following scenario happen? --> after the top Set variable completes can the bottom Set variable also complete BEFORE the value of the variable reaches the next activity in the top path (stuff on the right that is not shown here)? Let's say the activity that comes after the Set variable in the top path is a condition IF statement that expects a value of 1 and let's say that the activity that comes after the Set variable in the bottom path is a condition IF statement that expects a value of 2. If top completes first, then the variable value = 1. Then the bottom path completes, and the variable value = 2. Then the execution of the top path reaches a condition testing for IF variable = 1. But since the bottom path changed the value of the variable to 2, the top condition in the top path with evalute to false instead of true (remember that the top path Set variable was 1). Is this scenario possible? And if yes, is there a mechanism by which we can specify how the pipeline variables should be accessed , ie no shared memory, or some lock mechanism, etc..., instead of having to define 2 different variable names?Solved1.7KViews0likes3CommentsPIPELINE::revalidate your body expression to conform to the new editor experience
I add a Teams action onto the design surface of the pipeline, and as I am configuring it, this message displays above: Please revalidate your body expression to conform to the new editor experience. What's going on? What's there to validate as I just started plugging in the various values needed to for the Teams action to be able to send a message? Here's what it looks like (also I can save with no errors):923Views0likes3CommentsMicrosoft Fabric Data Pipeline + Power BI API Scanner
Hello, I'm looking to set up a data pipeline Fabric, I started by creating a copy data activity to retrieve a list of workspaces. But then I'm stuck, I can't manage to break down this list into packages of 100. Then for each block of 100 call the PostWorkspaceInfo API. Does anyone have any ideas? Best regards, Charline2.1KViews0likes5CommentsUnable to select lakehouse our warehouse in a pipeline (Copy data)
Hello everyone, I'm currently working on setting up a pipeline to transfer data from my Lakehouse to my Warehouse. However, I've encountered a difficulty: neither the Copy Assistant nor the Copy Canvas display the names of my "houses". I've attached an image below for further clarity: typo *appear Interestingly, after several page refreshes, the Lakehouse name finally appeared. However, when it came time to select the data destination, my Warehouse did not appear on the selection list. Then I refresh the page again and my Lakehouse disappeared from the list. Both Lakehouse and Warehouse are saved in myworkspace. Could this be a bug? If anyone has encountered this issue before, or if anyone has insights on how to remedy this situation, I would greatly appreciate your assistance. Thank you in advance for your time and help!2.5KViews0likes6Comments