Forum Discussion
Lakehouse to SQL Endpoint keeps giving error
I appreciate the help from you all. I atually endedup figuring out how to make it work.
I found the following code through some researchonto it and it ended up working. I think my columns had some spaces or something at the end.
from pyspark.sql.functions import col
def remove_bda_chars_from_columns(df: (
return df.select([col(x).alias(x.replace(" ", "_").replace("/", "").replace("%", "pct").replace("(", "").replace(")", "")) for x in df.columns])
SalesDocs_DF = SalesDocs_DF.transform(lambda df: remove_bda_chars_from_columns(df))
I appreciate the help from you all. I atually endedup figuring out how to make it work.
I found the following code through some researchonto it and it ended up working. I think my columns had some spaces or something at the end.
from pyspark.sql.functions import col
def remove_bda_chars_from_columns(df: (
return df.select([col(x).alias(x.replace(" ", "_").replace("/", "").replace("%", "pct").replace("(", "").replace(")", "")) for x in df.columns])
SalesDocs_DF = SalesDocs_DF.transform(lambda df: remove_bda_chars_from_columns(df))
Adding on to this for anyone that has a similar issue. This allows me to write a delta table without the column mapping so I can access the table through the SQL Endpoint.
Hi ghernandezmf ,
Glad to know that you were able resolve your issue.
This is awesome! Saved my life today, great solution, thank you!