Managed Delta Table Error

Does it work if you use this code below?

---------------------------------------------

from pyspark.sql.types import *

from pyspark.sql import functions as sf

from datetime import datetime

schema = StructType([

    StructField('id',IntegerType(), True),

    StructField('schema_name', StringType(), True),

    StructField('table_name', StringType(), True),

    StructField('watermark_value', TimestampType(), True),

    StructField('full_path', StringType(), True)

])

row_one = [

    (1, 'lorem', 'ipsum', datetime(1, 1, 1, 0, 0, 0), None),

]

df_one = spark.createDataFrame(row_one, schema)

df_two = df_one.withColumn('full_path', sf.concat(sf.col('schema_name'),sf.lit('.'),sf.col('table_name')))

df_two.show()

df_two.write.mode("overwrite").saveAsTable("watermark")

-----------------------------------------------

I don't think you need to specify %%pyspark as this is the default.

I don't think you need to initalize the spark session in your code in Fabric notebooks.

Maybe you need to add .mode("overwrite") or .mode("append") in the saveAsTable expression.

By the way, does your code run without errors if you remove line 28 in your code? (The saveAsTable line)