Forum Discussion

loic_bouscaud's avatar
loic_bouscaud
Regular Visitor
8 years ago
Solved

[M language] Transform column type to datetime with 2 string formats?

Hi all,

 

I have a column timestamp as string. In this column we can find two format of datetime dd/mm/yyyy hh:mm:ss (European datetime) and mm/dd/yyyy hh:mm:ss AM/PM (USA datetime).

I want to convert this column data type to datetime, but I can't with two string formats.

I tried to to make a custom column with an IF statement like this:

 

let
    Source = DocumentDB.Contents("https://avabot.documents.azure.com:443/"),
    history = Source{[id="history"]}[Collections],
    history_MessageHistoryItem = history{[db_id="history",id="MessageHistoryItem"]}[Documents],
    #"Expanded Document" = Table.ExpandRecordColumn(history_MessageHistoryItem, "Document", {"conversationid", "timestamp", "text", "id", "_rid", "_self", "_etag", "_attachments", "_ts", "sender", "recipient"}, {"Document.conversationid", "Document.timestamp", "Document.text", "Document.id", "Document._rid", "Document._self", "Document._etag", "Document._attachments", "Document._ts", "Document.sender", "Document.recipient"}),
    #"Convert Date" = Table.AddColumn(#"Expanded Document", "Custom Timestamp", each if Text.Contains([Document.timestamp], " AM") or Text.Contains([Document.timestamp], " PM") then Table.TransformColumnTypes(#"Expanded Document",{"Document.timestamp", type datetime},"en-US") else Table.TransformColumnTypes(#"Expanded Document",{"Document.timestamp", type datetime},null))
in
    #"Convert Date"

But i get a table in each rows, but the correct TransformColumnTypes is done for the row:

M query result

 

Then I tried to stock the string format in the custom column to use it in the transform column types statement:

let
    Source = DocumentDB.Contents("https://avabot.documents.azure.com:443/"),
    history = Source{[id="history"]}[Collections],
    history_MessageHistoryItem = history{[db_id="history",id="MessageHistoryItem"]}[Documents],
    #"Expanded Document" = Table.ExpandRecordColumn(history_MessageHistoryItem, "Document", {"conversationid", "timestamp", "text", "id", "_rid", "_self", "_etag", "_attachments", "_ts", "sender", "recipient"}, {"Document.conversationid", "Document.timestamp", "Document.text", "Document.id", "Document._rid", "Document._self", "Document._etag", "Document._attachments", "Document._ts", "Document.sender", "Document.recipient"}),
	#"Convert Date" = Table.AddColumn(#"Expanded Document", "Datetime Format", each if Text.Contains([Document.timestamp], " AM") or Text.Contains([Document.timestamp], " PM") then "en-US" else null),
	#"Changed Type" = Table.TransformColumnTypes(#"Convert Date",{"Document.timestamp", type datetime},[Datetime Format])
	
in
    #"Changed Type"

But i get this error: Expression.Error: There is an unknown identifier. Did you use the [field] shorthand for a _[field] outside of an 'each' expression?

I'm running out of idea, so if someone could help me with this issue I'll be very thankful to him.

Kind regards.

 

  • MarcelBeug's avatar
    MarcelBeug
    8 years ago

    You shouldn't replace the "_" at all.

     

    My intenton was to have you just follow the indicated steps and only adjust the generated code by adding the if .. then .. else part,

     

    But if you want to copy and adjust the code, then change the previous step name and the column reference.

    According to your example code, this would be:

     

    #"Convert Date" = Table.TransformColumns(#"Expanded Document",{{"Document.timestamp", each Date.From(DateTimeZone.From(_, if Text.Contains(_,"M") then "en-US" else "nl-NL")), type date}})

8 Replies

  • MarcelBeug's avatar
    MarcelBeug
    Community Champion

    Select the column, choose transform tab - Date - Parse and adjust the generated code to determine the culture (the if .. then ..else in the code below):

     

    = Table.TransformColumns(Source,{{"Document.Timestamp", each Date.From(DateTimeZone.From(_, if Text.Contains(_,"M") then "en-US" else "nl-NL")), type date}})

     

    • loic_bouscaud's avatar
      loic_bouscaud
      Regular Visitor

      Thanks MarcelBeug for your answer.

      I tried your to use your code and replaced the "_" by my timestamp column but I still got the same error.

      What should I use to replace the "_"?

      • MarcelBeug's avatar
        MarcelBeug
        Community Champion

        You shouldn't replace the "_" at all.

         

        My intenton was to have you just follow the indicated steps and only adjust the generated code by adding the if .. then .. else part,

         

        But if you want to copy and adjust the code, then change the previous step name and the column reference.

        According to your example code, this would be:

         

        #"Convert Date" = Table.TransformColumns(#"Expanded Document",{{"Document.timestamp", each Date.From(DateTimeZone.From(_, if Text.Contains(_,"M") then "en-US" else "nl-NL")), type date}})