I'm having trouble wrapping my head around this jerking back and forth motion here.

This has been super confusing...

This will help for the intuition behind gradient descent. The step size will influence the oscillation. https://www.youtube.com/watch?v=rIVLE3condE

Thanks! The video is very helpful