User Profile
x_mark_x
Advocate II
Joined 1 year ago
User Widgets
Contributions
Fabric IQ Ontology item for SaaS companies
Hello there Fabricators! I am starting to experiment with the Ontology item which sounds great on paper but I hit obsticles when it comes to defining the entities. The ontology demo that was presented at Microsoft Ignite was great, but as a data analyst in an SaaS company, i have a hard time to define our entities. In the demo example the business was aviation and airport related and the definition of the entities was quite straightforward, but for an SaaS company that sells licenses to customers with a valid from and valid to date the entity modeling is not so straightforward to me (not to mention that we have several types of licenses to several editions of our product and the customer can add or remove licenses at will...). Is there anyone out there who is currently trying to create the ontology for a license based SaaS company? If yes, it would be great to hear your thoughts, strategies and things you learned so far that can help me to create an ontology that can support a Data Agent and deliver trustworthy and consistent answers to the users. Please feel free to ask questions or add your two-cents to the topic 🙂Re: Semantic model refresh does not refresh data in report
Hi DuncanKing, Thanks for reaching out. I might have a solution now, but I need to verify its validity. It worked today at least...but I would need to confirm if this works consistently. On the other hand it would be nice to see if this method works with others as well. Here it goes. So I restructured my ETL pipeline first. Previously I had one main pipeline that was calling different notebooks for my bronze, silver and gold transformations, and connected the semantic model refresh activity after it. I am guessing that right after the last gold transformation notebook where I also save the data to my Lakehouse, there was not enough time for the Lakehouse metadata to be changed, and so the Semantic model refresh activity copied the old data. So, 1. I created child pipelines: - the bronze pipeline only held the copy activities from source and a bronze notebook (which calls an OAuth2 API, that I could not solve with any other activity, but this is a completely different issue) - a silver pipeline for the silver transformation notebooks - a gold pipeline for the gold transformations 2. I created a parent pipeline that calls the bronze, silver and gold pipelines after each other. 3. I created yet another pipeline that does the semantic model refresh. I followed the exact steps from this article to create this semantic model refresh pipeline: https://medium.com/@gordeyew/refresh-semantic-model-with-microsoft-fabric-data-pipeline-4f5e7f6c479b 4. I added the semantic model refresh pipeline after my gold pipeline (on success), AND made sure that the Wait on completion option was turned on on the Gold pipeline. Now my parent pipeline looks like this: NOTE: 1. I am also trying to implement CI/CD principles, and so far I only ran this pipeline in my dev stage as seen on the workspaces option. 2. - This morning I checked if the pipeline ran successfully, which it did. - Then I checked if I have the latest data in my Lakehouse, which it did. - Then I checked the semantic model if it was updated in the Refreshed column, which showed todays date. - Then I clicked the explore semantic model option and checked the transactionId column value to see if it is the same as in the Lakehouse, and I,ve seen the latest data. - Finally I've opened my report that was showing the old data, but when I refreshed the visual the report showed todays data. 3. This method needs to be validated to see if it is reliable or not. Verification will take me about a week or so... I will keep this post updated about my experience, and if you also end up trying this, please leave a feedback on your findings, it would be super helpful.
