Are there any matrix derivatives that DON'T look like the corresponding derivative if the matrix was replaced with variables (i.e. x and y), and if so, will we be seeing them in this course?
Why does the matrix have to be symmetric?
Will we ever have to differentiate an inverse matrix / how does that look like?
Is it useful to just know that they look like ordinary derivatives to help us remember the matrix derivatives? Or is there more to the relation that can help with understanding?
I have sometimes struggled with reasoning whether the derivative should be x or x^T (it's transpose). Do you have any intuition for this?