The issue with just following the gradients, however, is that we can get 'stuck' at a local maximum. There are solutions, some involving randomization, to mitigate this.

What types of "algorithms for optimization" might this be applied to? I understand the concept in general but am just curious for examples.

Gradients (and Hessians, for that matter) are used throughout optimization, from training convolutional neural networks to solving convex optimization problems. A classic "algorithm for optimization" is gradient descent (or ascent), which is the procedure being shown in this slide. The goal of this algorithm is to find the vector x that minimizes (or maximizes) the output of function f(x), subject to certain constraints. And gradients help you find this vector x.

As jfondrie mentioned, one issue with this procedure is that optimization algorithms can indeed get stuck for function f(x) with local peaks or valleys. There are, of course, a variety of solutions to handle these cases as well.