If we assume relatively smooth derivatives, will our speed improve if we take larger steps when the derivative is bigger? Additionally, could a special strategy be used when overshooting (when the derivative changes signs) to try to estimate the "midpoint" more accurately?
notme
The problem with overshooting strategies seems to be that we won't initially know if we overshot just the minimum or overshot twice, like in the graph.
ohmygearbox
how do people typically pick time step, # iterations, etc for their applications
PsychotiK
Not exactly sure how you would pick the time step initially but I'm assuming you can play around with it and if it jitters in your application then your time step could be too large. If it takes a very long time, then your time step could be too small.
linyingy
We can refer to the congestion control algorithm of TCP - additive-increase/multiplicative-decrease.
Simple is good enough for certain applications
If we assume relatively smooth derivatives, will our speed improve if we take larger steps when the derivative is bigger? Additionally, could a special strategy be used when overshooting (when the derivative changes signs) to try to estimate the "midpoint" more accurately?
The problem with overshooting strategies seems to be that we won't initially know if we overshot just the minimum or overshot twice, like in the graph.
how do people typically pick time step, # iterations, etc for their applications
Not exactly sure how you would pick the time step initially but I'm assuming you can play around with it and if it jitters in your application then your time step could be too large. If it takes a very long time, then your time step could be too small.
We can refer to the congestion control algorithm of TCP - additive-increase/multiplicative-decrease.