gradient-methods
FeaturedProblem-solving strategies for gradient methods in optimization
AI & Automation 552 stars
45 forks Updated 6 days ago MIT
Install
Quality Score: 94/100
Stars 20%
Recency 20%
Frontmatter 20%
Documentation 15%
Issue Health 10%
License 10%
Description 5%
Skill Content
# Gradient Methods
## When to Use
Use this skill when working on gradient-methods problems in optimization.
## Decision Tree
1. **Basic Gradient Descent**
- Update: x_{k+1} = x_k - alpha * grad f(x_k)
- Step size alpha: fixed, diminishing, or line search
- Convergence: O(1/k) for convex, linear for strongly convex
2. **Step Size Selection**
| Method | Approach |
|--------|----------|
| Fixed | alpha constant (requires tuning) |
| Backtracking | Armijo condition: f(x - alpha*grad) <= f(x) - c*alpha*||grad||^2 |
| Exact line search | minimize f(x - alpha*grad) over alpha |
| Adaptive | Adam, RMSprop (ML applications) |
3. **Accelerated Methods**
- Momentum: add velocity term
- Nesterov: look-ahead gradient
- Conjugate gradient: for quadratic functions
- `scipy.optimize.minimize(f, x0, method='CG')` - conjugate gradient
4. **Newton's Method**
- Update: x_{k+1} = x_k - H^{-1} * grad f
- Requires Hessian (expensive but quadratic convergence)
- Quasi-Newton (BFGS): approximate Hessian
- `scipy.optimize.minimize(f, x0, method='BFGS')`
5. **Convergence Diagnostics**
- Monitor ||grad f|| < tolerance
- Check function value decrease
- Watch for oscillation (step size too large)
- `sympy_compute.py diff "f" --var x` for gradient
## Tool Commands
### Scipy_Bfgs
```bash
uv run python -c "from scipy.optimize import minimize; res = minimize(lambda x: (x[0]-1)**2 + 100*(x[1]...
Details
- Author
- vibeeval
- Repository
- vibeeval/vibecosystem
- Created
- 8 months ago
- Last Updated
- 6 days ago
- Language
- C#
- License
- MIT
Integrates with
Similar Skills
Semantically similar based on skill content — not just same category
AI & Automation Featured
gradient-methods
Problem-solving strategies for gradient methods in optimization
3,831 Updated 1 weeks ago
foryourhealth111-pixel AI & Automation Featured
gradient-methods
Problem-solving strategies for gradient methods in optimization
3,951 Updated 2 days ago
parcadei AI & Automation Listed
ml-optimization-methods
优化方法选型:拿到优化问题先问"凸不凸/有无约束/梯度贵不贵"。用户问"牛顿法还是梯度下降/ L-BFGS 用在哪/KKT 有什么用/SVM 为何转对偶"、纠结学习率收敛诊断、担心陷局部最优时激活。 动作: 凸性判定→一阶vs二阶→拉格朗日与KKT对偶→收敛曲线归因→高维鞍点重估。 不适用于: 训练排障(ml-deep-training-playbook)、横向选型(ml-task-matching)。 trigger: convex optimization 凸优化, L-BFGS, KKT conditions, duality 对偶, 鞍点
1 Updated 2 months ago
fieldlu