Forum Discussion

SergioPerdiz's avatar
SergioPerdiz
Frequent Visitor
2 years ago
Solved

Shortcut shows different data

I have a shortcut in the silver lakehouse to a table in the bronze lakehouse. However it seems like the data is not updated.   When I look at the SQL endpoint of the bronze lakehouse I get the bel...
data engineering
Lakehouse
Notebook
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi SergioPerdiz,

    I think this should be related to the setting which define the ‘data time zone’, by default datetime values will be stored with UTC format.

    For the first one , your SQL statement get the records from table. For the second one that run in notebook, you have set the time zone offset which may affect the datetime values.

    Regards,

    Xiaoxin Sheng

Anonymous's avatar
Anonymous
Not applicable
2 years ago

Hi SergioPerdiz,

I think this should be related to the setting which define the ‘data time zone’, by default datetime values will be stored with UTC format.

For the first one , your SQL statement get the records from table. For the second one that run in notebook, you have set the time zone offset which may affect the datetime values.

Regards,

Xiaoxin Sheng

  • SergioPerdiz's avatar
    SergioPerdiz
    Frequent Visitor
    2 years ago

    Hi Anonymous,

     

    Thanks, that was indeed the issue!

    Then I have another question how can I handle the following.

    The notebook is executed around 01:00 AM GMT +2, which is 11:00 PM UTC, however for some calculations I use the below code. 

     

    # Get today's date 
today_date = current_date().cast("string")

     

    So that means that for last night's run it would return 2024/07/11, but it should be 2024/07/12.

    A solution could be to change the code to the below, but I'm wondering if this would be the correct/best way to do it.

     

    # Get today's date 
today_date = (current_date() +1).cast("string")

     

    • Anonymous's avatar
      Anonymous
      Not applicable
      2 years ago

      HI SergioPerdiz,

      If you had to calculate with specific time zone datetime, you can add a new column to the table to calculate with the offset between UTC and local time zone. Than you can use the new column to work as local datetime.

      Regards,

      Xiaoxin Sheng