Bridging Continuous Learning and Discrete Optimisation in Artificial Intelligence
Artificial intelligence has always had two halves. One searches, plans, schedules and proves: it is discrete, exact and verifiable, and its native language is combinatorics. The other perceives, predicts and generalises from data: today that means neural networks, which are continuous, statistical and trained by gradients, and whose native language is calculus. After decades of oscillating between the two, the field has settled on hybrid approaches. A navigation app feeds learnt journey times into a route planner, a language model invokes a tool and reads back the result, and a demand forecast determines which power stations will operate tomorrow.
Every such system contains a link between a continuous learner and a discrete decision-maker. What matters is not the quality of either part on its own, but the result produced by the system as a whole. The continuous part is therefore trained end-to-end, so that it learns to make proper use of the discrete part. The standard method for such training is gradient descent: nudge the parameters in the direction of a better final decision, and repeat. Through the discrete component, however, this breaks down: a decision either remains unchanged or changes abruptly, and neither tells the learner which way to move.
We will examine the few principled methods for obtaining a useful update direction nonetheless, ranging from straightforward techniques and reinforcement learning to smoothed surrogates, implicit differentiation and differentiable simulation, and consider what each of them sacrifices in return. The navigation app, the tool-using model and the power grid all depend on the answer.