Forum Discussion

roalexan's avatar
roalexan
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago
Solved

create Lakehouse table using pyspark

Hello all. At a high level, I'm trying to create an EventStream that reads from an EventHub as a streaming source, and writes to a Fabric Lakehouse as a destination (using this blog sample, BTW). Foc...
one lake
  • Anonymous's avatar
    Anonymous
    2 years ago

    Hi roalexan 
    Thanks for using Fabric Community.
    1) Please attach the lakehouse to the notebook.


    2) Then try running this code:

    from pyspark.sql.types import StructType, StructField, IntegerType, StringType
schema = StructType([
    StructField("Id", IntegerType(), False),
    StructField("Name", StringType(), True)
])
df = spark.createDataFrame([], schema)
df.write.format("delta").saveAsTable("table2")


    This will create a new table named "table2" in your lakehouse.

     

    Hope this helps. Please let me know if you have any further questions. Glad to help.

roalexan's avatar
roalexan
Icon for Microsoft Employee rankMicrosoft Employee
2 years ago

thanks! The solution was indeed to use code such as above

 

df.write.format("delta").saveAsTable("table2")

 

I will take a closer look at doc such as here. Wasn't sure delta format was strictly required.

  • Anonymous's avatar
    Anonymous
    Not applicable
    2 years ago

    Hi roalexan 
    Glad that your query got resolved. Lakehouse currently supports only delta format tables while creating a new table. Link1

    Please continue using Fabric Community for any help regarding your queries.