palantir-migration-deep-dive

# Palantir Migration Deep Dive ## Overview Comprehensive guide for migrating data into Foundry, migrating from legacy systems to Foundry-backed architectures, and upgrading between Foundry API versions using the strangler fig pattern. ## Prerequisites - Source system access and schema documentation - Foundry enrollment with write access - Understanding of Foundry data pipeline architecture (`palantir-reference-architecture`) ## Instructions ### Step 1: Migration Assessment ```markdown ## Migration Checklist - [ ] Source system inventory (tables, volumes, refresh rates) - [ ] Data classification (PII, confidential, public) - [ ] Schema mapping: source columns → Foundry dataset columns - [ ] Volume estimate: rows, GB, growth rate - [ ] Dependencies: downstream consumers of source data - [ ] Timeline: parallel run period, cutover date ``` ### Step 2: Data Migration — Bulk Import ```python import foundry, pandas as pd client = get_foundry_client() # Read source data (example: PostgreSQL) df = pd.read_sql("SELECT * FROM orders WHERE year >= 2024", source_conn) # Upload to Foundry dataset client.datasets.Dataset.upload( dataset_rid="ri.foundry.main.dataset.xxxxx", branch_id="master", file_path="orders.parquet", data=df.to_parquet(), content_type="application/x-parquet", ) print(f"Uploaded {len(df)} rows to Foundry") ``` ### Step 3: Incremental Sync (Ongoing) ```python from datetime import datetime, timedelta def incremental_sync(client, source_conn, da...