Optimization Interview Prep
Optimization interview prep — convex optimization, KKT conditions, duality, gradient and Newton methods, interior-point, SGD, and applications to ML and engineering.
Quick answer
Optimization is the mathematical theory of finding extrema of functions, possibly subject to constraints.
Optimization questions test whether candidates can recognize problem structure and exploit it.
Editorial review
Written by
CompoundLearn editorial team
Wireless / RF / hardware engineering
Reviewed by
CompoundLearn editorial team
Wireless / RF / hardware engineering
Last reviewed
Built from curated topic maps, editorial validation, and subject-matter review so the page stays aligned with the interview intent and the current content pipeline.
What it is
Optimization is the mathematical theory of finding extrema of functions, possibly subject to constraints. Standard topics include unconstrained optimization theory (first-order necessary conditions, second-order sufficient conditions), convex sets and functions and the rich structure of convex optimization, gradient descent and its convergence analysis on smooth and strongly-convex objectives, Newton's method and quasi-Newton variants (BFGS, L-BFGS), line search and trust-region globalization strategies, constrained optimization and the KKT conditions, Lagrangian duality and its consequences (weak and strong duality, dual problems, dual decomposition), specific problem classes (linear programming, quadratic programming, semidefinite programming, conic programming), interior-point methods and their polynomial-time complexity, stochastic optimization and SGD, accelerated methods (Nesterov momentum, Adam, AdamW), proximal methods for non-smooth objectives (ISTA, FISTA, ADMM), constrained-set projections and projected-gradient methods, and an introduction to non-convex optimization (escape from saddle points, basin analysis, global optimization heuristics like simulated annealing). For engineering candidates, optimization is the algorithmic core of machine learning (training is a non-convex optimization), control theory (model predictive control solves a QP at every timestep), signal processing (compressed sensing is an L1-regularized least-squares problem), and operations research (LPs and IPs everywhere).
Why interviewers ask
Optimization questions test whether candidates can recognize problem structure and exploit it. ML interviews probe gradient-based methods (why does SGD converge despite noise, when does momentum help, what does Adam adapt that vanilla SGD does not), the bias-variance interpretation of regularization (L2 = ridge = MAP with Gaussian prior, L1 = lasso = sparse MAP with Laplace prior), and second-order methods (Newton for natural gradient, K-FAC for approximate Fisher). Controls interviews use QPs in MPC and probe whether candidates can derive the dual or recognize when warm-starting saves real-time computation. Compressed-sensing and signal-processing interviews use L1 minimization and probe restricted-isometry conditions and proximal algorithms (ISTA, FISTA). The strongest candidates recognize convexity by inspection, know which solver is appropriate for which problem class (LP solver, QP solver, conic solver, generic non-linear), articulate convergence rates (sublinear for plain GD on smooth non-convex, linear on smooth strongly-convex, quadratic for Newton near the optimum, accelerated O(1/k^2) for Nesterov on smooth convex), and quote KKT conditions as the source of all optimality structure. Interviewers also use optimization to test candidates on the gap between mathematical results and engineering practice — a candidate who can derive the OLS estimator but does not know that ill-conditioning makes the closed form numerically unstable signals classroom knowledge without practitioner depth.
Common mistakes
The most common mistake is failing to recognize convex problems and applying general non-convex methods to them — convex problems have efficient global solvers, but candidates who do not check convexity end up running heuristics on what should be a polynomial-time problem. A second mistake is misapplying the KKT conditions: forgetting to check the constraint qualification (LICQ, Slater), or treating KKT as sufficient when the problem is non-convex (KKT is necessary but not sufficient there). A third mistake is using gradient descent on ill-conditioned problems without preconditioning — the convergence rate is O((κ - 1)/(κ + 1))^k, which is glacial for κ in the thousands; preconditioning, second-order methods, or accelerated methods are the right responses. A fourth mistake is conflating descent with optimality — gradient descent decreases the objective at each step but does not reach the optimum at any finite k; "convergence" requires explicit step-size schedules or stopping criteria, and stopping early gives an approximate solution whose error must be quantified. A fifth mistake is on duality: candidates derive the dual but do not realize that strong duality requires Slater's condition (or a similar regularity assumption) and that arbitrarily-defined duals give lower bounds but not equalities. A sixth mistake is using Newton without globalization — Newton diverges or oscillates far from the optimum, and a line search or trust region is mandatory in production code. A seventh mistake is misunderstanding stochastic optimization: candidates apply standard convergence rates from deterministic analysis to SGD without accounting for the variance term, or use a fixed learning rate where decay (or adaptive methods) is needed. An eighth mistake is on regularization: candidates pick L2 by default without thinking about whether sparsity (L1) or group sparsity (L2,1) is the right inductive bias for the problem. Finally, candidates often confuse the primal and dual roles: the dual of a minimization is a maximization, and the dual variables (multipliers) have specific interpretations as sensitivities of the optimum to constraint perturbations — useful in shadow-price arguments in operations research and in interior-point algorithm design.
Frequently asked questions
- What are the KKT conditions and when are they necessary or sufficient?
- For a constrained optimization problem (minimize f subject to g_i ≤ 0 and h_j = 0), the KKT conditions at an optimum x* require: stationarity of the Lagrangian (∇f + Σ μ_i ∇g_i + Σ λ_j ∇h_j = 0), primal feasibility, dual feasibility (μ_i ≥ 0), and complementary slackness (μ_i · g_i = 0). KKT are necessary at any local minimum that satisfies a constraint qualification (LICQ, Slater). For convex problems with Slater's condition, KKT are also sufficient — any KKT point is a global minimum.
- Why is convexity such a strong property?
- A convex objective on a convex feasible set has the property that every local minimum is global, and any descent direction reduces the objective. This collapses the optimization landscape: there are no spurious local minima, no saddle points to escape, and gradient-based methods converge to the optimum given sufficient steps. Convex problems often admit polynomial-time solutions (linear programs, quadratic programs, semidefinite programs all have efficient solvers), while general non-convex problems are NP-hard. Recognizing convexity is therefore one of the most valuable skills in optimization.
- What is duality and why does it matter?
- For every constrained optimization (primal), there is an associated dual problem built in two stages: for fixed multipliers μ ≥ 0 and λ, first minimize the Lagrangian over the primal variable x — an unconstrained infimum over all x, since the constraints are folded into the Lagrangian rather than restricting x to feasible points — to get the dual function g(μ, λ) = inf_x L(x, μ, λ); then maximize g over the multipliers. Weak duality says the dual gives a lower bound on the primal optimum. Strong duality says they are equal — guaranteed for convex problems with Slater's condition. Duality lets you solve the easier of the two problems (the dual is sometimes much smaller), provides certificates of optimality, and yields algorithms like the SVM dual that exploit kernel structure unavailable in the primal.
- What is the difference between gradient descent, Newton, and quasi-Newton?
- Gradient descent steps in the direction -∇f with step size α: linear convergence on smooth strongly-convex problems, slow on ill-conditioned problems. Newton uses the Hessian: x_{n+1} = x_n - H^(-1) ∇f, quadratic convergence near the optimum, but each step costs O(n^3) for the Hessian solve and O(n^2) memory. Quasi-Newton (BFGS, L-BFGS) approximates the Hessian from secant information, achieving superlinear convergence at much lower per-step cost — the workhorse for medium-scale smooth optimization.
- How do interior-point methods solve LPs and SDPs?
- Interior-point methods (IPMs) maintain strict feasibility throughout the iteration by adding a logarithmic barrier to the objective: minimize f(x) - μ Σ log(g_i(x)). As μ shrinks toward 0, the barrier-modified solution traces a path (central path) toward the true optimum. Newton's method on the perturbed KKT system gives polynomial-time complexity. IPMs are the standard solver for LPs (Karmarkar broke the simplex monopoly in 1984), QPs, and SDPs (where they are essentially the only practical approach).
- What is stochastic gradient descent and why does it work for ML at scale?
- SGD replaces the full gradient ∇f(x) with an unbiased estimate from a mini-batch. Each step is O(batch size) instead of O(dataset size). Convergence is in expectation, with variance proportional to the noise in the gradient estimate. SGD works at ML scale because it amortizes data-pass cost, fits naturally into streaming and online settings, and the implicit noise actually helps escape saddle points and shallow minima in non-convex landscapes. Variants (momentum, Adam, AdamW) adapt step size per parameter and combine with weight decay for the modern training recipe.
Related topics
Essential AI-Native Skills for Optimization
Modern engineering work increasingly uses AI tools for design and code review, debugging, documentation, test and testbench generation, and workflow automation. The goal is not to let AI replace engineering judgment — it is to move faster while keeping verification discipline.
- Use AI to explain unfamiliar code, logs, waveforms, datasheets, or test failures.
- Break large problems into small, reviewable steps you can verify independently.
- Ask AI for hypotheses, then validate them against tests, measurements, simulations, or lab data.
- Version-control your analysis scripts, testbenches, and configs — keep changes small and reviewable.
- Document your assumptions, design tradeoffs, and debugging decisions.
- Verify AI output before trusting it: run the checks that fit the domain — unit tests, linters, simulations, or bench/lab measurements.
- Review AI output for correctness, edge cases, and real-world consequences.
Optimization — coming to the question bank
The adaptive practice engine is already live for core wireless, RF, and ML systems. Optimization isn't covered in the question bank yet — get notified when it's added.