Forum Discussion

frithjof_v's avatar
frithjof_v
Community Champion
2 years ago
Solved

External table not available in SQL Analytics Endpoint or Power BI

Hi,   I used a Notebook to create an external table (unmanaged table) in my Lakehouse based on a txt file in my Lakehouses' File folder:   %%sql CREATE TABLE testtxtexternal (     name STRING,...
data engineering
Lakehouse
Notebook
  • TomMartens's avatar
    TomMartens
    2 years ago

    Hey frithjof_v ,

     

    the reason why the table does not appear is the format of the source file. The lingua franca for everything Microsoft Fabric is the Delta Table format. If you want to shortcut to the txt file, then you need a step that reads the txt file into a dataframe and writes the dataframe as a Delta Table.

     

    Hopefully, this helps to tackle your challenge.

     

    Regards,

    Tom

frithjof_v's avatar
frithjof_v
Community Champion
2 years ago

Thank you TomMartens,

 

I guess that means we will need to trigger (or schedule) some ETL tool (e.g. Data pipeline, Dataflow Gen2, Notebook, etc.) if we want to bring in updated data from the .txt file into SQL Analytics Endpoint (as it requires the data to be in Delta table format).

 

Perhaps we can use Storage Event Trigger to trigger the ETL each time the .txt file gets updated in ADLS Gen2, and update the Delta table.

 

Best regards, Frithjof