Should the camera keep its "up" direction as the camera being transformed? If you only pick an "up" direction ad hoc when you need to do the projection, how do you make sure your roll is consistent between different projections?