Is the only reason a really small h is worse because of floating point rounding?
When referring to relative error in the graph, what is the error "relative" to? If we were to plot the absolute error, instead of seeing the error shoot back up, would we instead get a plateau?
So how do we pick h? Do we just try many different values and see where the difference converges?
Why does the error function seen in the graph change so quickly? I would think that for "reasonable" functions, the impact of changing h slightly would be relatively small.
Do we just test bunch of h and pick the one with smallest error?
Is it possible to optimally pick h based on our function or do we have to sort of "spray and pray" with several different h values?
How would we determine if h is "too small?" And regarding relative error, what are we saying the error is relative to?
If we don't know the correct/true derivative of a function, how would we know the relative error?
Does h change very drastically from function to function or is there a default value that could work decently for a majority (or a certain subset)?
How can we know when we have a relative error if we have no idea what the actual function looks like?
Is this similar to aliasing where if the value you choose is too small/different, it would cause the values to be incorrect.