Convex Optimization & KKT

Lagrange multipliers, KKT optimality conditions, Slater condition, and duality.

For a small example, minimize the one-dimensional objective x^2 + 2x + 1. Complete the square or follow the derivative to x = -1 and verify the curve has one global minimum.

In practice

Choose weights with a risk objective and a fixed budget. Write the convex objective and constraints, solve it, then inspect dual values to understand binding limits.

Initializing Interactive Playground...

Experiment notes

Record your experiment

Predict how one control changes the Convex Optimization & KKT result.

PredictionWhat changed, and why?

Chapter 1 of 7 · Core

Convex Sets, Functions & Epigraph Geometry

In mathematical optimization, a set is defined as convex if the line segment connecting any two points in lies entirely within .

Mathematical Definition of a Convex Set:

Mathematical Definition of a Convex Function: A function is convex if its domain is a convex set and for all and :

Epigraph & First-Order Characterization: A continuously differentiable function is convex if and only if its epigraph is a convex set, which yields the global first-order lower-bound inequality:

This implies that for a convex function, any local minimum is guaranteed to be a global minimum!

Second-Order Conditions: A twice-differentiable function is convex if and only if its Hessian matrix is positive semi-definite for all :

Key equations

PyTorch Convexity & Hessian Eigenvalue Inspectorpython

import torch

def inspect_function_convexity(func, x_val: torch.Tensor):
    """Computes the Hessian matrix of a scalar function f(x) and checks positive semi-definiteness."""
    x_val = x_val.clone().detach().requires_grad_(True)
    y = func(x_val)
    
    # Compute Hessian matrix via autograd
    hessian = torch.autograd.functional.hessian(func, x_val)
    eigenvalues = torch.linalg.eigvalsh(hessian)
    
    is_convex = torch.all(eigenvalues >= -1e-6)
    print(f"Hessian Eigenvalues at x={x_val.tolist()}: {eigenvalues.tolist()}")
    print(f"Is function locally convex? {is_convex}")
    return is_convex

# Example: Quadratic objective f(x) = x1^2 + 3*x2^2 + 2*x1*x2
f_quadratic = lambda x: x[0]**2 + 3*x[1]**2 + 2*x[0]*x[1]
inspect_function_convexity(f_quadratic, torch.tensor([1.0, 2.0]))

NextLagrangian Duality & Weak vs Strong Duality

References

Primary sources and free technical reading for Convex Optimization & KKT.