It depends on wether you push the camera by the back or the front.

For aircraft, most people imagine pushing the tail around, so stick down is pitch up.

For looking, most people imaging pushing the field of view (the eyes), so stick up is look up.