Forum Discussion

Koritala's avatar
Koritala
Post Patron
1 month ago
Solved

How to Convert String time column to timestamp or DATE DATATYPE using

Hi,

I am peforming the data load from lake house table to Silver table where one of the date column in the existing table.

As Silver tables not support to load string type date column values to DATE/TIMESTAMP in silver layer table,  is there any conversion function to do this convertion? If you can provide one example by taking one date (string type column) to convert into DATE/TIMESTAMP will appriciate.

Thanks,

Sri.

  • If you're working in Fabric using Spark or a notebook, you can convert a string column to either a DATE or TIMESTAMP using the built-in conversion functions.

    For example, if your source column contains values such as '2026-07-02':

    SQL

    SELECT

    TO_DATE(date_string_col, 'yyyy-MM-dd') AS converted_date

    FROM source_table;

    Show more lines

    If the source value includes time, such as '2026-07-02 14:30:00', use:

    SQL

    SELECT

    TO_TIMESTAMP(date_string_col, 'yyyy-MM-dd HH:mm:ss') AS converted_timestamp

    FROM source_table;

    Show more lines

    In PySpark:

    Python

    from pyspark.sql.functions import to_date, to_timestamp

     

    df = df.withColumn(

    "DateColumn",

    to_date("StringDateColumn", "yyyy-MM-dd")

    )

     

    df = df.withColumn(

    "TimestampColumn",

    to_timestamp("StringTimestampColumn", "yyyy-MM-dd HH:mm:ss")

    )

    Show more lines

    Make sure the format string matches the actual values in your source column. If the source data contains formats such as MM/dd/yyyy or dd-MM-yyyy, adjust the pattern accordingly. You can then write the transformed data into your Silver table with the target column defined as DATE or TIMESTAMP.

     

  • Hi Koritala,

    You can't directly load a string into a DATE/TIMESTAMP column — you need to cast it during the SELECT/transform step before writing to Silver. Just make sure the source string format matches the pattern you pass to the conversion function.

    Spark SQL

    SELECT
      to_date(order_date_str, 'yyyy-MM-dd')             AS order_date,
      to_timestamp(order_ts_str, 'yyyy-MM-dd HH:mm:ss') AS order_ts
    FROM bronze.orders

    PySpark

    from pyspark.sql.functions import to_date, to_timestamp
    
    silver_df = (
        bronze_df
        .withColumn("order_date", to_date("order_date_str", "yyyy-MM-dd"))
        .withColumn("order_ts",   to_timestamp("order_ts_str", "yyyy-MM-dd HH:mm:ss"))
    )
    
    silver_df.write.mode("append").saveAsTable("silver.orders")


    If your source format is different (e.g., dd/MM/yyyy or MM-dd-yyyy HH:mm), adjust the pattern accordingly — otherwise you'll get NULLs. Exact syntax can vary slightly by platform (Fabric / Databricks / Synapse), so if you share your source string format and target platform, I can tailor it further.

    Hope this helps!

    If this got you what you needed, a Kudos and an Accepted Solution mark would be great — it helps others searching for the same thing find the answer quicker.

    Disclosure: My earlier response above was drafted with the assistance of an AI tool (Microsoft Copilot) and cross-checked against Microsoft Learn documentation. Sharing this note for transparency as per the Community AI Usage Policy

  • Hi Koritala 

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.


    Thank you.

6 Replies

  • If you're working in Fabric using Spark or a notebook, you can convert a string column to either a DATE or TIMESTAMP using the built-in conversion functions.

    For example, if your source column contains values such as '2026-07-02':

    SQL

    SELECT

    TO_DATE(date_string_col, 'yyyy-MM-dd') AS converted_date

    FROM source_table;

    Show more lines

    If the source value includes time, such as '2026-07-02 14:30:00', use:

    SQL

    SELECT

    TO_TIMESTAMP(date_string_col, 'yyyy-MM-dd HH:mm:ss') AS converted_timestamp

    FROM source_table;

    Show more lines

    In PySpark:

    Python

    from pyspark.sql.functions import to_date, to_timestamp

     

    df = df.withColumn(

    "DateColumn",

    to_date("StringDateColumn", "yyyy-MM-dd")

    )

     

    df = df.withColumn(

    "TimestampColumn",

    to_timestamp("StringTimestampColumn", "yyyy-MM-dd HH:mm:ss")

    )

    Show more lines

    Make sure the format string matches the actual values in your source column. If the source data contains formats such as MM/dd/yyyy or dd-MM-yyyy, adjust the pattern accordingly. You can then write the transformed data into your Silver table with the target column defined as DATE or TIMESTAMP.

     

  • Hi Koritala,

    You can't directly load a string into a DATE/TIMESTAMP column — you need to cast it during the SELECT/transform step before writing to Silver. Just make sure the source string format matches the pattern you pass to the conversion function.

    Spark SQL

    SELECT
      to_date(order_date_str, 'yyyy-MM-dd')             AS order_date,
      to_timestamp(order_ts_str, 'yyyy-MM-dd HH:mm:ss') AS order_ts
    FROM bronze.orders

    PySpark

    from pyspark.sql.functions import to_date, to_timestamp
    
    silver_df = (
        bronze_df
        .withColumn("order_date", to_date("order_date_str", "yyyy-MM-dd"))
        .withColumn("order_ts",   to_timestamp("order_ts_str", "yyyy-MM-dd HH:mm:ss"))
    )
    
    silver_df.write.mode("append").saveAsTable("silver.orders")


    If your source format is different (e.g., dd/MM/yyyy or MM-dd-yyyy HH:mm), adjust the pattern accordingly — otherwise you'll get NULLs. Exact syntax can vary slightly by platform (Fabric / Databricks / Synapse), so if you share your source string format and target platform, I can tailor it further.

    Hope this helps!

    If this got you what you needed, a Kudos and an Accepted Solution mark would be great — it helps others searching for the same thing find the answer quicker.

    Disclosure: My earlier response above was drafted with the assistance of an AI tool (Microsoft Copilot) and cross-checked against Microsoft Learn documentation. Sharing this note for transparency as per the Community AI Usage Policy

  • v-csrikanth's avatar
    v-csrikanth
    Community Support

    Hi Koritala 

    We would like to inquire whether have you got the chance to check the solutions provided by Dev_Dholakia sannavajjala to resolve the issue. We hope the information provided helps to clear the query. Should you have any further queries, kindly feel free to contact the Microsoft Fabric community

    Thanks,
    Srikanth Cheri

    Community Support Team

  • v-csrikanth's avatar
    v-csrikanth
    Community Support

    Hi Koritala 

    We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. And, if you have any further query do let us know.


    Thank you.

  • Hi!  What are you using to do the transformation? Notebook or Gen2DataFlow?