palantir-data-handling

# Palantir Data Handling ## Overview Handle sensitive data in Foundry using markings (data classifications), column-level security, PII redaction in transforms, and GDPR/CCPA deletion workflows. ## Prerequisites - Foundry enrollment with Markings enabled - Understanding of your organization's data classification policy - Familiarity with transforms (`palantir-core-workflow-a`) ## Instructions ### Step 1: Data Classification with Markings Foundry Markings control who can access data at the dataset, column, or row level. | Marking | Access | Use Case | |---------|--------|----------| | `PUBLIC` | All users | Aggregated reports, reference data | | `INTERNAL` | Employees only | Business metrics, operational data | | `CONFIDENTIAL` | Specific groups | Customer PII, financial data | | `RESTRICTED` | Named individuals | Compensation, legal, M&A | ### Step 2: PII Redaction in Transforms ```python from transforms.api import transform_df, Input, Output from pyspark.sql import functions as F @transform_df( Output("/Company/datasets/customers_safe"), customers=Input("/Company/datasets/raw_customers"), ) def redact_pii(customers): """Create an analytics-safe view with PII removed.""" return ( customers .withColumn("email", F.sha2(F.col("email"), 256)) # Hash email .withColumn("phone", F.lit("***-***-****")) # Mask phone .withColumn("ssn", F.lit(None).cast("string")) # Remove SSN .with...