palantir-performance-tuning

# Palantir Performance Tuning ## Overview Optimize Foundry API performance: efficient pagination, client-side caching, batch object retrieval, and Spark transform tuning with `@configure` profiles. ## Prerequisites - Completed `palantir-install-auth` setup - Working Foundry integration to optimize - Access to Foundry build metrics (for transform tuning) ## Instructions ### Step 1: Efficient Pagination ```python from functools import lru_cache def fetch_all_objects(client, ontology: str, object_type: str, page_size: int = 500): """Fetch all objects with maximum page size to minimize API calls.""" all_objects = [] page_token = None while True: result = client.ontologies.OntologyObject.list( ontology=ontology, object_type=object_type, page_size=min(page_size, 500), # Foundry max is 500 page_token=page_token, ) all_objects.extend(result.data) page_token = result.next_page_token if not page_token: break return all_objects ``` ### Step 2: Client-Side Caching ```python from cachetools import TTLCache import hashlib, json _cache = TTLCache(maxsize=1000, ttl=300) # 5-minute TTL def cached_get_object(client, ontology, object_type, primary_key): """Cache Ontology object reads to reduce API calls.""" cache_key = f"{ontology}:{object_type}:{primary_key}" if cache_key in _cache: return _cache[cache_key] obj = client.ontologies.OntologyObj...