We are going to move from SQL server to Snowflake as our target database for the warehouse. Today we have most of our ETL development done in ODI (Oracle Data Ingegrator).

So I'm intressted in to know if anyone is using ODI together with Snowflake and how it's woking. And what experince/recommendations you have of other ETL tools together with Snowflake as target.

For example

• Matillion
• DBT
• Xplenty

Today we have started with using NIFI moving the data from source to Azure blob storage. But we are not sure if ODI is the right tool for the rest when we are in the cloud.

I'm really looking forward to see all your answers

CodePudding user response：

Snowflake supports both transformations during (ETL) or after loading (ELT).

Snowflake works with a wide range of data integration tools, including Informatica, Talend, Tableau, Matillion and others.

In data engineering, new tools and self-service pipelines are eliminating traditional tasks such as manual ETL coding and data cleaning companies. With easy ETL or ELT options via Snowflake, data engineers can instead spend more time working on critical data strategy and pipeline optimization projects.

With a Snowflake as your data lake and data warehouse, ETL can be effectively eliminated, as no pre-transformations or pre-schemas are needed.

In addition, Snowflake Snowpark is designed to make building complex data pipelines a breeze and to allow developers to interact with Snowflake directly without moving data. Read more about Snowpark here.

https://www.snowflake.com/trending/etl-tools

CodePudding user response：

If you started to transfer data from the source to Azure blob storage, I assume that you have a subscription in Azure and it is possible that Snowflake itself is placed in the Azure region.

In this case, I recommend using Azure Data Factory directly, so you have everything on one provider and support for data migration from SQL Server.

Link to documentation: Copy and transform data in Snowflake using Azure Data Factory