hyperparameter-tuning

# Hyperparameter Tuning This skill enables an AI agent to systematically search for optimal hyperparameter configurations for machine learning models. It covers defining search spaces, selecting search strategies (grid, random, Bayesian, Hyperband), running trials with cross-validation, applying early stopping to prune poor configurations, and analyzing results to identify the best-performing parameters. The agent balances exploration and exploitation to find strong configurations within a given computational budget. ## Workflow 1. **Define the search space:** Specify each hyperparameter with its type (categorical, integer, float) and range. Use log-uniform distributions for parameters that span orders of magnitude (e.g., learning rate from 1e-5 to 1e-1). Group related parameters and define conditional search spaces where certain parameters only apply when others take specific values. 2. **Select the search strategy:** Choose the tuning algorithm based on compute budget and search space size. Grid search is exhaustive but only feasible for small spaces. Random search is a strong baseline that scales better. Bayesian optimization (Tree-structured Parzen Estimators or Gaussian Processes) is most sample-efficient for expensive evaluations. Hyperband and ASHA combine early stopping with random search for deep learning workloads. 3. **Configure evaluation:** Set up k-fold cross-validation (typically 5-fold) for reliable performance estimates on small to medium datasets. For l...