Hello Community,
We are working on our first real time data visual and are in need of some advise/help.
We are currently running a simulation on Azure cloud compute. While the event is running some data is outputed for every iteration into a log file. We would like to be able to view the log file in Power BI as a real time visual both in a dashboard and in a report. Steps we have currently taken are creating a Azure streaming job, then created the output for Powe BI.
The log file created is a bit dirty and needs some clean up before we can use it. Is there a cloud based way to do this clean up?
Currently we are sending/downloading the log file to a local machine, cleaning it with PowerQuery and some Python script, saving it as a .CSV file and sending it back up to Azure. Is there a better way to do this?
As a note, we do not have any option to change the initial output log file from the simulation. 😞
Thanks for any help!!
