databricks-common-errors

# Databricks Common Errors ## Overview Quick-reference diagnostic guide for the most frequent Databricks errors. Covers cluster failures, Spark OOM, Delta Lake conflicts, permissions, schema mismatches, rate limits, and job run failures with real SDK/SQL solutions. ## Prerequisites - Databricks CLI configured - Access to cluster/job logs - `databricks-sdk` installed for programmatic debugging ## Instructions ### Step 1: Identify the Error Source ```bash # Get failed run details databricks runs get --run-id $RUN_ID --output json | jq '{ state: .state.result_state, message: .state.state_message, tasks: [.tasks[] | {key: .task_key, state: .state.result_state, error: .state.state_message}] }' ``` ### Step 2: Match and Fix --- ### CLUSTER_NOT_READY / INVALID_STATE ``` ClusterNotReadyException: Cluster 0123-456789-abcde is not in a RUNNING state ``` **Cause:** Cluster is starting, terminating, or in error state. ```python from databricks.sdk import WorkspaceClient from databricks.sdk.service.compute import State w = WorkspaceClient() cluster = w.clusters.get(cluster_id="0123-456789-abcde") if cluster.state in (State.PENDING, State.RESTARTING): w.clusters.ensure_cluster_is_running("0123-456789-abcde") elif cluster.state == State.TERMINATED: w.clusters.start_and_wait(cluster_id="0123-456789-abcde") elif cluster.state == State.ERROR: reason = cluster.termination_reason print(f"Cluster error: {reason.code} — {reason.parameters}") # Common: CLOUD_PROV...