Forum Discussion
Real Time log analytics data in PowerBI
No problem, always happy to help.
You have mentioned the limitations that I was thinking of. There is no underlying database, hence no filtering possibilites. As mentioned by Microsoft: "With a streaming dataset, there is no underlying database, so you cannot build report visuals using the data that flows in from the stream. As such, you cannot make use of report functionality such as filtering, custom visuals, and other report functions." Real-time streaming will be an overkill here.
In your case simplest Direct Query will do the job. Try setting up a small Azure SQL Database. Then schedule a job of pulling log data and pushing it into the DB - I do not know how to do it at the moment, but there are probably multiple ways. Later connect to the DB using Direct Query in PowerBI. This way the data will always be up-to-date, as PowerBI constantly queries the DB for the data. The only limitation here is the volume of the data - a report might not be as responsive, as we would like. In your case with 1000s of rows, this should not be a problem.
Let me know, what you think of this. In 2-3 posts, we might have a whole architecture ready :D
You can use a "push dataset" and the Power BI back-end will keep up to 5 million rows, and yes, you can filter on it, too, and create visuals.
I recently wrote an article with source code to a sample Azure Function and submitted to my company's Blog Keepers. When it gets posted I will let you know.
I might be able to send you a copy of the Word doc if you're interested. PM me.
Thanks,
== Ross ==
Hi Ross,
I'm interested for the sample Azure Function.
I don't really understand how to pull the log analytics data using Functions, since it's quite a new thing to me.
I came across this thread when looking for pushing real time data of log analytics to PowerBI and interested to try it with Functions and Stream Analytics.
Is the article available now?
