Need Help with solving ODBC - Gateway OutOfMemoryError in service

[Anonymous]

Hi, 

I am using an ODBC data source and configured incremental refresh in service. When I ran the first refresh, I get an error as below after it ran for 50 min. The dataset has 100+ million rows of data in the table.

Autogenerated below code in Power query for ODBC datasource. 

let
Source = Odbc.DataSource("dsn=LRS-Dev-MongoDB", [HierarchicalNavigation=true]),
learninglocker_Database = Source{[Name="learninglocker",Kind="Database"]}[Data],
statements_Table = learninglocker_Database{[Name="statements",Kind="Table"]}[Data],

.........

It would be of great help if someone can assist what i can do to make this to work.

[GilbertQ]

Hi @Anonymous 

The error is saying that it is consuming too much memory.

What I would suggest doing is to rather use an actual query to get your data, instead of what you have done above. Your code shown above does not have anything for the incremental part with the rangeStart and rangeEnd?

[Anonymous]

Hi @GilbertQ ,

I have changed it to odbc query and with a simple select statement, the query loadss the preview data but with a left outer join i need to use, i get an error below.

The query I am passing is:

Should I reach out to the DB team to expose a single table/view with all the columns I needed by joining all the tables? I can then do a star schema model to separate out fact & dim tables...

Any suggestions in overcoming the error above or alternate options to proceed?

[GilbertQ]

Hi @Anonymous 

That is an issue from the database, so I would suggest chatting to your DBA and see how they can assist with a query to get the data out as efficiently as possible.