snowflake-core-workflow-a

# Snowflake Core Workflow A — Data Loading ## Overview Primary data loading workflow: stages, file formats, COPY INTO, and Snowpipe for continuous ingestion. ## Prerequisites - Completed `snowflake-install-auth` setup - Target table created in Snowflake - Source data in S3, GCS, Azure Blob, or local files - Role with `CREATE STAGE` and `USAGE` on warehouse ## Instructions ### Step 1: Create a File Format ```sql -- CSV format CREATE OR REPLACE FILE FORMAT my_csv_format TYPE = 'CSV' FIELD_DELIMITER = ',' SKIP_HEADER = 1 NULL_IF = ('NULL', 'null', '') EMPTY_FIELD_AS_NULL = TRUE FIELD_OPTIONALLY_ENCLOSED_BY = '"' ERROR_ON_COLUMN_COUNT_MISMATCH = FALSE; -- JSON format (for semi-structured data) CREATE OR REPLACE FILE FORMAT my_json_format TYPE = 'JSON' STRIP_OUTER_ARRAY = TRUE IGNORE_UTF8_ERRORS = TRUE; -- Parquet format CREATE OR REPLACE FILE FORMAT my_parquet_format TYPE = 'PARQUET' SNAPPY_COMPRESSION = TRUE; ``` ### Step 2: Create a Stage ```sql -- External stage (S3) CREATE OR REPLACE STAGE my_s3_stage STORAGE_INTEGRATION = my_s3_integration URL = 's3://my-bucket/data/' FILE_FORMAT = my_csv_format; -- External stage (GCS) CREATE OR REPLACE STAGE my_gcs_stage STORAGE_INTEGRATION = my_gcs_integration URL = 'gcs://my-bucket/data/' FILE_FORMAT = my_csv_format; -- Internal stage (Snowflake-managed storage) CREATE OR REPLACE STAGE my_internal_stage FILE_FORMAT = my_csv_format; -- List files in stage LIST @my_s3_stage; ``` ### Step...