Forum Discussion
Transforming Data Through SQL Endpoint for Enhanced Power BI Dashboards
Hi Ashwath_Bala_S
Thanks for explaining your ask further.
1) You are right about the first approach.
2)The process cannot be automated. But you can run the notebooks at desired time intervals. After the data gets refreshed in Dataverse, you can schedule to run notebook which would run on daily basis.
You can also use the notebook activity which can be scheduled for desired time intervals. Please refer to this document for more information : Link1
Based on both your approaches, the first option is better one.
Hope this helps. Please let us know if you have any further questions.
Hi,
Thanks for your response.
The FnO tables is brought into Dataverse. From Dataverse, the tables are linked in Microsoft Fabric (Data present in Dataverse, linked to fabric as shortcut tables).
First approach, bringing the data from Lakehouse into SQL End-point, creating a visual query, removing nulls, creating calculated columns and then saving as a view and then building the report on that. (I believe, once data is updated in Dataverse, gets updated through the view in "Report").
Secondly, using Spark Notebook, loading a dataframe, making transformations and then saving them as a "Delta" table in OneLake, I am uncertain if that will be automated, once if data is refreshed in Dataverse.
Finally, having a warehouse, I would prefer other methods, since data is already in Lakehouse and in the Warehouse, I need to use the same Visual Query and Measure to do the transformations. (Since, data is already in Lakehouse, generally prefer it, but, need to check on the same).
The flow I prefer is like once it is refreshed in D365 F&O, linked with Dataverse, the same should be automated on the report in Fabric. Any assistance for the automated (Either SQL-EndPoint or Spark Notebooks or Data Warehouse) will be helpful.
I hope I have got your points, if not kindly assist, since Fabric is a fresh perspective
Thanks in Advance.
Hi Ashwath_Bala_S
Thanks for explaining your ask further.
1) You are right about the first approach.
2)The process cannot be automated. But you can run the notebooks at desired time intervals. After the data gets refreshed in Dataverse, you can schedule to run notebook which would run on daily basis.
You can also use the notebook activity which can be scheduled for desired time intervals. Please refer to this document for more information : Link1
Based on both your approaches, the first option is better one.
Hope this helps. Please let us know if you have any further questions.