Forum Discussion

SnoekL's avatar
SnoekL
Icon for Helper II rankHelper II
2 years ago
Solved

DataFlow Gen2 issue We couldn't convert to Date.

I'm loading data from a on-premise Gupta database using ODBC with DataFlow Gen2

It's working fine until I try to make it an incremental refresh using the last date from the already loaded table.

 

I'm extracting the data from the warehouse with:

This seems to be working fine.

I'm using a native query and I use the output of the first query as a value (VW_INCR_DATE) in the sql native query

In the dataflow (when editing) it seems to be working. The results are showing correct. 

But the issue shows up when the dataflow runs.

I don't understand this issue. Is anyone able to help me with this issue?

 

 

 

 

dataflow
  • SnoekL's avatar
    SnoekL
    2 years ago

    The VW_INCR_DATE value is coming from the warehouse and is the last date for which the data is already loaded. That is not from the source.

  • SnoekL's avatar
    SnoekL
    2 years ago

    I found the solution. The query which extracts the Date value from the warehouse should have "enable staging" false to get it working. Not sure why though :).

6 Replies

  • SnoekL's avatar
    SnoekL
    Icon for Helper II rankHelper II
    2 years ago

    I found the solution. The query which extracts the Date value from the warehouse should have "enable staging" false to get it working. Not sure why though :).

    • Cosmic_Panda's avatar
      Cosmic_Panda
      Frequent Visitor
      2 years ago

      Thanks a lot.

       

      It was really giving me a hard time. Can't believe it was simply turning off "Enable Staging". 

    • SnoekL's avatar
      SnoekL
      Icon for Helper II rankHelper II
      2 years ago

      aj1973, the topic you are referring to is actually the one I used to build my incremental refresh.

      Big difference though is that the example has sql database as a source where I'm connecting via odbc to a local database. 

      And the example has a datekey in the source which is a integer where my source only has a datetime field to use to get the incremental data.

       

      I tried to convert the source data to different datatypes but than I get different errors as well.  So i'm stuck at the moment.

      • aj1973's avatar
        aj1973
        Icon for Community Champion rankCommunity Champion
        2 years ago

        When connecting to ODBC

        Add a SQL statement to convert/cast this part 

        to Date before loading into Power Query Online.

         

        Power Query couldn't accept to add a step to Format that column so try with SQL statement, it should work.