Forum Discussion

lakshmipai's avatar
lakshmipai
Regular Visitor
2 years ago

Snowflake Iceburg Table

Hi Team,

I am trying to create Iceburg Table in snowflake for that I am trying to create External volume,to connect to the Microsoft Fabric..I am not able to find the options while creating New DataBase in Snowflake for enabling of iceburgtables option as mentioned in the below document.
So is it publicly available or still in preview.

 

Snowflake & Microsoft: Enhancing Interoperability with Apache Iceberg

6 Replies

  • Anonymous's avatar
    Anonymous
    Not applicable

    Hi lakshmipai 
    Thanks for using Fabric Community.

    Creating Iceberg tables in Snowflake involves several steps:

    Create Users and Roles:
    You need to have the necessary permissions to create and manage Iceberg tables.
    How to Create Iceberg Tables in Snowflake | phData

    Create an External Volume in Snowflake:
    An external volume is a named, account-level Snowflake object that you use to connect Snowflake to your external cloud storage for Iceberg tables. For instructions on creating an external volume, you can refer to the Snowflake documentation.
    Getting Started with Iceberg Tables (snowflake.com)
    Create an Iceberg table | Snowflake Documentation

    Create Iceberg Tables in Snowflake:
    You can create an Iceberg table by using the CREATE ICEBERG TABLE command. To define table columns, you can use Iceberg data types.
    Snowflake & Microsoft: Enhancing Interoperability with Apache Iceberg
    Iceberg Tables in Snowflake. Iceberg Tables in Snowflake are in… | by Feng Li | Snowflake Builders Blog: Data Engineers, App Developers, AI/ML, & Data Science | Apr, 2024 | Medium

    As for your question about the availability of Iceberg tables in Snowflake, as of the latest information, Iceberg Tables in Snowflake are in public preview. Please note that the availability status might change, and it’s always a good idea to check the latest Snowflake documentation or announcements for the most up-to-date information.

    Hope this helps. Please let me know if you have any further questions.

  •  

    i have done a project in which i transfer the data from sql server to power bi report and the step by step process is explained below
    1) first i create the pipeline in fabric data factory though which i transfer the data from sql server to fabric data lake which layer i called bronze layer . i created two different pipeline first is full and second is delta . if i want to transfer the all data then i need to execute the full otherwise i execute delta in which the change tracking is updated in the table which is stored in sql server .
    2) then whatever the full and delta data are stored in the bronze layer i transfer this into silver by writing the pyspark notebook and i create only one table for one data set like in this i didnot create separate full or delta table if i run the notebook by passing the in_mode para as full then it will truncate the whole data of the table from silver and add the wholle data from full table whihc is in bronze similarly for delta para only updated values are updataed , inserted , deleted
    3) After that i want to transsfer the data from fabric silver to blob though pipline which i created in fabric |
    4) then to transfer the data from blob storage to snowflake i use the concept of iceburge .

     


    i have done till blob storage but my doubt is how i can and why i create the iceberge to transfer the data from blob to snowflake  

    OTHERWISE IS IT POSSIBLE TO LOAD DATA FROM FABRIC SILVER TO DIRECT SNOWFLAKE SILVER LAYER