Connecting PowerBI directly to Rapid7 via API

[KL008]

Hi all,

I'd like to create a direct connection between Rapid 7's Vulnerability Scanning tool and PowerBI. 

I have some questions regarding the limitations:

1. Can we see historical data this way, or for this do I have to log it in a database/use a CSV

2. Can it be fully automated using scheduled refreshes?

3. How much data before it becomes impractical? 

Regards,

K

[Kedar_Pande]

Power BI doesn’t automatically keep historical data, so you’ll need to store it in a database or export it as CSV files if you want to track changes over time. The good news is that it can be automated using Power BI’s scheduled refresh feature, as long as the connection is solid

However, if you’re working with a lot of data, Power BI can start to slow down, especially with millions of rows. In that case, you might need to optimize your data or consider using Power BI Premium for better performance.

Your Kudos/Likes are much appreciated!

If this post helps, please consider Accepting it as the solution to help the other members find it more quickly.

Regards,

Kedar Pande

www.linkedin.com/in/kedar-pande

[Anonymous]

Hi @KL008 ,

Regarding questions 1 and 2, I recommend using the Import mode with incremental refresh, as this will ensure that historical data is retained and refreshed regularly. You need to ensure that the API supports pagination and provides timestamps so that incremental refresh can correctly retrieve new or changed data. For question 3, it's advisable to keep the dataset size below 1GB.

Best Regards,

Bof

[KL008]

Thanks! When it comes to the file size, once I've done the initial import will the file size only include the data included in the incremental refresh, or will the file size be all the data?

Finally, can you explain what import mode is, I'm quite the newbie

[Anonymous]

Hi @KL008 ,

I think the solution described in this blog should meet your requirements. Please refer to it:

https://sqldusty.com/2020/05/20/power-bi-incremental-refresh-with-web-api-data-source/

Best Regards,

Bof

[KL008]

In the example you sent he doesn't keep the historical data, he drops all data from over 7 days prior. Is it possible to not do this, or is he only doing this with the data he is importing from the SQL databset and the rest is retained in PBI?

Also, is the SQL database required in order to do the query folding, or can I skip out on this altogether?