DM_GWPipeline_Gateway_MashupDataAccessError

Hello. I have setup a virtual machine with a on premises gateway installed, it's in a vnet aswell.

Connection with gateway from PowerBI.com is successful, and I can run refresh manually without any problems.

I'm having issues with scheduled refresh. It's really inconsistent, one day it can run 8 refreshes successfully, the other day you have 2 successfull, then 1 fail, then 2 successful, then 3 fails. 

The errors messages I'm receiving are:

Data source error: {"error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_MashupDataAccessError","parameters":{},"details":[{"code":"DM_ErrorDetailNameCode_UnderlyingErrorCode","detail":{"type":1,"value":"-2147467259"}},{"code":"DM_ErrorDetailNameCode_UnderlyingErrorMessage","detail":{"type":1,"value":"An error happened while reading data from the provider: 'Internal .NET Framework Data Provider error 6.'"}},{"code":"DM_ErrorDetailNameCode_UnderlyingHResult","detail":{"type":1,"value":"-2147467259"}},{"code":"Microsoft.Data.Mashup.ValueError.DataSourceKind","detail":{"type":1,"value":"SQL"}},{"code":"Microsoft.Data.Mashup.ValueError.DataSourcePath","detail":{"type":1,"value":"sqlserver;DV"}},{"code":"Microsoft.Data.Mashup.ValueError.Reason","detail":{"type":1,"value":"DataSource.Error"}}],"exceptionCulprit":1}}} Table: table

Data source error: {"error":{"code":"DM_GWPipeline_Gateway_DataSourceAccessError","pbi.error":{"code":"DM_GWPipeline_Gateway_DataSourceAccessError","parameters":{},"details":[],"exceptionCulprit":1}}} Table: table.

Setup:

I have the latest update of PowerBI Desktop installed and also the gateway.

It's Azure SQL Server.

PowerBI file has parameters on the connection string, but only two connections in datasource settings, which are the two databases in the sql server the gateway is connected to. 

Searched all over google, but found no solution. Any ideas?