tutorial requests
56 TopicsRefresh Semantic Model - Table Dynamic Content
I've set up dynamic content inside of a Refresh Semantic Model activity inside of a data pipeline. So far I've dynamically set the Workspace Id and the Semantic Model but I've previously left the table line blank. The pipeline would refresh properly until now and the only way it seems to run even with static Workspace Id and Semantic Model values is if I specify the tables. For the static values I can wait for the drop down menu to load and select all but when I use dynamic content in Workspace and Semantic Model ID, the table names won't load and I can only input dynamic content. How do I write in the tables through dynamic content. I've tried the following based off the JSON file: {"table": "TableName"} TableName table:TableName [{"table": "TableName"}] TableName1, TableName2 Editing the JSON file directly by copying and pasting how a non-dynamic content inputted Workspace and Semantic model ID has the table names set up How do I dynamically write in multiple tables for the dynamic content? Is there a way to specify select all when I write in dynamic content?Solved5.4KViews1like7CommentsFilter CSV dataset based on a column
Hi, I am a newbie to ADF. i have CSV dataset referring from SFTP holding 14 million records. Currently, it has copyactivity which copies to CSV dataset(unzipped) from source zipped format dataset! Then dataflow , which refers this CSV and loads to AzureSQL (upsert operation) . The pipeline fails at times. Its an ask to filter read data only from 8 days backdated based on a column "DateColumn" present in the CSV file. I added this condition(filter) in dataflow (Source ->filter ->sink) where Filter takes expression as DateColumn>= toString(addDays(currentDate(), -8)). The pipeline doesnt haveconsistent success! any other method would help since the data would be growing for years .Solved809Views0likes4CommentsIs there an ETA on event based trigger for pipeline (specially integration with sharepoint)?
Hi, I was wondering if there is a way to trigger a pipeline when a file at a sharepoint point location is modifed/created without using REST API and power shell? I have been using power automate to do this but would like to remove this dependency. If this trigger is not yet present in fabric pipeline....is there an ETA when it will be ? ThankyouSolved1.5KViews1like4CommentsHow to retrieve the output of a look-up activity in a notebook
Hi, I have a pipeline in which I perform a lookup on a table and retrieve this: { "count": 7, "value": [ { "WorkspaceID": "WKS13", "WorkspaceName": "Test Workspace 5", "UserEmail": "email" }, { "WorkspaceID": "WKS9", "WorkspaceName": "Test Workspace 9", "UserEmail": "email" }, { "WorkspaceID": "WKS9", "WorkspaceName": "Test Workspace 9", "UserEmail": "email" }, { "WorkspaceID": "WKS12", "WorkspaceName": "Test Workspace 12", "UserEmail": "email" }, { "WorkspaceID": "WKS11", "WorkspaceName": "Test Workspace 11", "UserEmail": "email" }, { "WorkspaceID": "WKS15", "WorkspaceName": "Test Workspace 11", "UserEmail": "email" }, { "WorkspaceID": "WKS16", "WorkspaceName": "Test Workspace 11", "UserEmail": "email" } ] } And I would like to retrieve this in a notebook. Do you have a solution?Solved2.6KViews0likes7CommentsStore Activity Data into a Table
I am wanting to store the data collected from the Get Metadata activity into a lakehouse table, along with the Copy Data activity output for each table loaded in the ForEach activity. I have tried using variables of type array, and have now been testing out system variables. I haven't had any luck with figuring it out so far. Does anyone have any suggestions?Solved3.2KViews1like8CommentsHow to Send Alerts to a Security Group Instead of Individual Emails in an MS Fabric Pipeline
Hello community, I currently have a pipeline in Microsoft Fabric that sends email alerts based on a specific use case. To manage these alerts, I use a configuration table that contains the alert type and the email addresses associated with it. So far, the process works through a stored procedure that queries this table and, depending on the alert frequency, sends an email to the configured recipients. However, I now want to modify this logic to send alerts to a security group instead of individual email addresses. My idea is that email addresses would be associated with a security group, and by sending the alert to the group, all its members would automatically receive the notification. My questions are: - Is it possible to retrieve the email addresses associated with a security group within Microsoft Fabric? - What would be the best way to integrate this functionality into my pipeline so that the stored procedure can handle security groups instead of fixed email lists? I appreciate any guidance or recommendations on how to implement this efficiently. Thanks in advance for your help!Solved2.4KViews0likes6CommentsSAP Connection with Select Incremental
Hello, good day. I am looking for a way to ingest SAP data into Fabric with the feature of an incremental select, that is, to ingest only the new/modified data, I saw that it is possible to do it with a dataflow but you want to minimize costs so I want to know if it can be done with a pipeline and how? or if there are other options I would greatly appreciate the information.Solved1.3KViews0likes3CommentsHow to do incremental refresh in data pipeline?
Hi all, I would like to perform a 24-hour incremental refresh based on the last modified data. I found some details from online sources and Microsoft documentation, but I am unable to find specific information for data pipelines. Does anyone know how to do this?Solved6.6KViews1like5CommentsIssues configuring a pipeline in Fabric to obtain the email list of a security group
Hi community, I am working with a pipeline in Fabric to obtain the email list of the members of a security group. I have already configured the necessary web activities to obtain the token, generated through a service principal. This token is used in a "get_emails" activity to get the email list. However, this "get_emails" activity requires creating a connection using the URL of the security group, but it doesn't work. Could you help me identify the problem and suggest solutions to correctly configure the connection? im using this URL to get de list --> https://graph.microsoft.com/v1.0/groups/GroupID/members This is the settings section for the "get_emails" activitieSolved1.7KViews0likes7CommentsAutomation of Monitoring Scheduled Pipelines and Notebooks
Hi Community, I need help creating a notebook that can perform the following activity: Currently, the pipelines and some notebooks are executed because we have manually scheduled them to run daily, weekly, etc., at certain times, with a start date and an end date. When the end date arrives (we have set several years in advance), the pipeline execution will stop. At the moment, it is not possible to set an infinite end date so that the execution never stops. I would like to create a pipeline that can search for all pipelines and notebooks across all workspaces, and for those with an active schedule, check when the end date is and send alerts when there are 1 month, 2 months, 3 months left until their end date, so that the execution period can be extended if necessary. I have a Client ID and a Secret ID for authentication. Could you help me with this?Solved2KViews0likes4Comments