AWS + Snowflake: Building Modern Data and AI Workflows with Glue, S3, Iceberg, and Cortex

Modern data platforms are no longer just about moving data from one database to another. Organizations need scalable data pipelines, governed data access, open table formats, real-time analytics, and AI-powered insights.

Two platforms that play an important role in this ecosystem are Amazon Web Services (AWS) and Snowflake.

AWS provides a broad ecosystem for storage, data integration, security, analytics, and machine learning. Snowflake provides a cloud data platform for data warehousing, data sharing, analytics, and AI-powered applications.

The interesting part is how these platforms can work together without forcing organizations to move all their data into a single environment.

In this article, we'll explore important AWS and Snowflake integration capabilities and build a practical mini-project that moves sales data from Amazon S3 through AWS Glue into Snowflake.

Many organizations already store large volumes of data in Amazon S3 while using Snowflake for analytics and business intelligence.

Combining these platforms can help teams:

A simplified architecture looks like this:

Data Sources | v Amazon S3 (Raw Data) | v AWS Glue (ETL Processing) | v Snowflake (Data Warehouse) | v SQL Analytics BI / AI Apps Enter fullscreen mode Exit fullscreen mode Each service has a specific responsibility, allowing teams to build flexible data architectures.

One important development is the ability to access Snowflake Horizon Catalog metadata through AWS Glue catalog federation.

This is particularly useful when organizations work with Apache Iceberg tables and want to discover and query data across platforms.

Traditionally, integrating separate data catalogs can require additional metadata synchronization and operational complexity.

Catalog federation helps reduce that friction.

The key idea is that teams can access catalog metadata without treating every dataset as a separate copy that must be moved into another system.

Always verify the supported authentication method, catalog configuration, and table compatibility for your specific environment.

AWS Glue can connect to Snowflake to support ETL workloads.

Authentication and credential management are important parts of this integration.

For production environments, prefer supported OAuth-based authentication or securely managed credentials over hardcoding passwords in scripts.

Keep credentials in an appropriate secret-management solution and grant only the permissions required by the job.

Another interesting direction is the integration of Amazon Quick with Snowflake Cortex AI through the Model Context Protocol (MCP).

This brings conversational analytics closer to business users.

Instead of manually writing every query, users can ask questions about their data using natural language.

Snowflake Cortex capabilities can support different AI workflows, including structured-data analysis, document search, and agent-based interactions.

Amazon Quick can provide a conversational interface for interacting with supported Snowflake capabilities.

This creates opportunities for more accessible analytics while keeping data access and governance important parts of the architecture.

Apache Iceberg is an open table format designed for large analytical datasets.

For organizations using both AWS and Snowflake, Iceberg can help reduce dependence on proprietary storage formats and improve interoperability.

Amazon S3 | v Apache Iceberg Tables | +------+------+ | | v v AWS Glue Snowflake | | v v Athena Analytics Enter fullscreen mode Exit fullscreen mode The exact integration pattern depends on catalog ownership, permissions, supported table features, and the query engine being used.

Let's build a simple data engineering project.

Project objective: Read sales data from Amazon S3, transform it using AWS Glue, load it into Snowflake, and run analytical queries.

sales.csv | v Amazon S3 | v AWS Glue Job (PySpark ETL) | v Snowflake SALES_DB | v SALES_ANALYTICS Enter fullscreen mode Exit fullscreen mode Step 1: Create a sample CSV file Create a file named sales.csv:

order_id,customer,product,quantity,price,region 1001,Ali,Laptop,1,1200,Asia 1002,Sara,Mobile,2,600,Europe 1003,Ahmed,Tablet,1,400,Asia 1004,Ayesha,Laptop,2,1200,North America 1005,Usman,Mobile,1,600,Asia 1006,Hina,Monitor,2,300,Europe Enter fullscreen mode Exit fullscreen mode Step 2: Upload the file to Amazon S3 Create an S3 bucket and upload the CSV file.

s3://my-data-engineering-bucket/raw/sales.csv Enter fullscreen mode Exit fullscreen mode Your bucket should contain:

my-data-engineering-bucket/ | └── raw/ └── sales.csv Enter fullscreen mode Exit fullscreen mode Step 3: Create a Snowflake database and table Run the following SQL in Snowflake:

CREATE DATABASE SALES_DB; CREATE SCHEMA SALES_DB. ANALYTICS; CREATE TABLE SALES_DB. ANALYTICS. SALES ( ORDER_ID NUMBER, CUSTOMER VARCHAR ( 100 ), PRODUCT VARCHAR ( 100 ), QUANTITY NUMBER, PRICE NUMBER ( 10, 2 ), REGION VARCHAR ( 100 ), TOTAL_AMOUNT NUMBER ( 12, 2 ) ); Enter fullscreen mode Exit fullscreen mode Step 4: Configure AWS Glue Before running the job:

The exact connector configuration depends on your AWS Glue version and the Snowflake connector setup you choose.

The following example demonstrates the transformation logic.

from pyspark.sql import SparkSession from pyspark.sql.functions import col spark = SparkSession. builder \. appName ( " SalesETLPipeline " ) \. getOrCreate () # Read CSV from Amazon S3 df = spark. read \. option ( " header ", True ) \. option ( " inferSchema ", True ) \. csv ( " s3://my-data-engineering-bucket/raw/sales.csv " ) # Calculate total sales amount df = df. withColumn ( " TOTAL_AMOUNT ", col ( " quantity " ) * col ( " price " ) ) # Remove records with missing required values df = df. dropna ( subset = [ " order_id ", " customer ", " product ", " quantity ", " price " ] ) # Display transformed data df. show () # Inspect schema df. printSchema () Enter fullscreen mode Exit fullscreen mode This example focuses on the transformation itself. To complete the pipeline, configure the Snowflake connector's supported Spark write integration and connection options, then write the transformed DataFrame into SALES_DB.ANALYTICS.SALES.

Do not assume that a generic JDBC or connector configuration will work unchanged across every Glue runtime. Follow the connector documentation for the selected runtime.

Once the transformed records are loaded, run:

SELECT SUM ( TOTAL_AMOUNT ) AS TOTAL_REVENUE FROM SALES_DB. ANALYTICS. SALES; Enter fullscreen mode Exit fullscreen mode Revenue by region

SELECT REGION, SUM ( TOTAL_AMOUNT ) AS REVENUE FROM SALES_DB. ANALYTICS. SALES GROUP BY REGION ORDER BY REVENUE DESC; Enter fullscreen mode Exit fullscreen mode Top-selling products by revenue

SELECT PRODUCT, SUM ( TOTAL_AMOUNT ) AS TOTAL_SALES FROM SALES_DB. ANALYTICS. SALES GROUP BY PRODUCT ORDER BY TOTAL_SALES DESC; Enter fullscreen mode Exit fullscreen mode Customer purchase analysis

SELECT CUSTOMER, COUNT ( * ) AS TOTAL_ORDERS, SUM ( TOTAL_AMOUNT ) AS TOTAL_SPENT FROM SALES_DB. ANALYTICS. SALES GROUP BY CUSTOMER ORDER BY TOTAL_SPENT DESC; Enter fullscreen mode Exit fullscreen mode You now have a basic cloud data pipeline using Amazon S3, AWS Glue, PySpark, and Snowflake.

A working demo is only the first step. Production data platforms require additional planning.

The best architecture depends on where your data lives, which platform owns the catalog, your governance requirements, and the workloads you need to support.

AWS and Snowflake are not necessarily competing choices. They can complement each other in a modern data architecture.

AWS provides scalable infrastructure, data ingestion, transformation, storage, and governance capabilities. Snowflake provides a powerful environment for analytical workloads, data sharing, and AI-powered data experiences.

With catalog federation, Apache Iceberg interoperability, and conversational analytics integrations, organizations have more options for building connected data platforms.

The practical takeaway is simple: choose the right service for each workload, keep governance at the center, and avoid unnecessary data movement.

Start with a small pipeline, measure its performance and cost, and expand the architecture as your data requirements grow.

If you're building data pipelines with AWS and Snowflake, try the mini-project and extend it with incremental loads, data-quality checks, and automated orchestration.