For the first question, $\mathbf{n} \times (\mathbf{n} \times \mathbf{u}) = -\mathbf{u}$.
Haboric
For second question, let's assume that the rotated vector is $\mathbf{u}^{\prime}$. We can project $\mathbf{u}^{\prime}$ onto $\mathbf{u}$ and $\mathbf{n} \times \mathbf{u}$ to find the coefficients that are needed to represent $\mathbf{u}^{\prime}$ in terms of $\mathbf{u}$ and $\mathbf{n} \times \mathbf{u}$. Projecting $\mathbf{u}^{\prime}$ onto $\mathbf{u}$, we can find the scaled coefficient along $\mathbf{u}$:
$$\frac{\left|\mathbf{u}^{\prime}\right|cos(\theta)}{\left|\mathbf{u}\right|} = \frac{\left|\mathbf{u}\right|cos(\theta)}{\left|\mathbf{u}\right|} = cos(\mathbf{\theta})$$
Similarly, projecting $\mathbf{u}^{\prime}$ onto $\mathbf{n} \times \mathbf{u}$, we have
$$\frac{\left|\mathbf{u}^{\prime}\right|sin(\theta)}{\left|\mathbf{n}\times\mathbf{u}\right|} = \frac{\left|\mathbf{u}\right|sin(\theta)}{\left|\mathbf{n}\times\mathbf{u}\right|}$$.
Then
$$\mathbf{u}^{\prime} = cos(\mathbf{\theta}) \mathbf{u} + \frac{|\mathbf{u}|sin(\theta)}{|\mathbf{n}\times\mathbf{u}|} \mathbf{n}\times \mathbf{u}$$
keenan
@Haboric Right, this is the right idea: I can easily construct a basis $\mathbf{e}_1 := \mathbf{u}/|\mathbf{u}|$ and $\mathbf{e}_2 := \mathbf{n} \times \mathbf{u}/|\mathbf{n} \times \mathbf{u}|$ for the plane. From there, a rotation of $\vec{u}$ by an angle $\theta$ is just
For the first question, $\mathbf{n} \times (\mathbf{n} \times \mathbf{u}) = -\mathbf{u}$.
For second question, let's assume that the rotated vector is $\mathbf{u}^{\prime}$. We can project $\mathbf{u}^{\prime}$ onto $\mathbf{u}$ and $\mathbf{n} \times \mathbf{u}$ to find the coefficients that are needed to represent $\mathbf{u}^{\prime}$ in terms of $\mathbf{u}$ and $\mathbf{n} \times \mathbf{u}$. Projecting $\mathbf{u}^{\prime}$ onto $\mathbf{u}$, we can find the scaled coefficient along $\mathbf{u}$: $$\frac{\left|\mathbf{u}^{\prime}\right|cos(\theta)}{\left|\mathbf{u}\right|} = \frac{\left|\mathbf{u}\right|cos(\theta)}{\left|\mathbf{u}\right|} = cos(\mathbf{\theta})$$ Similarly, projecting $\mathbf{u}^{\prime}$ onto $\mathbf{n} \times \mathbf{u}$, we have $$\frac{\left|\mathbf{u}^{\prime}\right|sin(\theta)}{\left|\mathbf{n}\times\mathbf{u}\right|} = \frac{\left|\mathbf{u}\right|sin(\theta)}{\left|\mathbf{n}\times\mathbf{u}\right|}$$.
Then $$\mathbf{u}^{\prime} = cos(\mathbf{\theta}) \mathbf{u} + \frac{|\mathbf{u}|sin(\theta)}{|\mathbf{n}\times\mathbf{u}|} \mathbf{n}\times \mathbf{u}$$
@Haboric Right, this is the right idea: I can easily construct a basis $\mathbf{e}_1 := \mathbf{u}/|\mathbf{u}|$ and $\mathbf{e}_2 := \mathbf{n} \times \mathbf{u}/|\mathbf{n} \times \mathbf{u}|$ for the plane. From there, a rotation of $\vec{u}$ by an angle $\theta$ is just
$$\tilde{u}(\theta) := |\vec{u}|(\cos(\theta)\vec{e}_1 + \sin(\theta)\vec{e}_2).$$