Accumulating rotations for camera

You must maintain each of your three local axes. When you rotate around one, you must update the other two.

class gameclass { Vector3 forward = Vector3.UnitZ; //persistent orientation variables Vector3 left = -1 * Vector3.UnitX; Vector3 up = Vector3.UnitY Matrix world = Matrix.Identitiy; InputClass inputclass; //something to get your input data void Update() { Vector3 pitch = inputclass.getpitch(); //vertical swipe/stick forward = Vector3.transform(forward, Matrix.CreateFromAxisAngle(left, pitch)); up = Vector3.transform(up, Matrix.CreateFromAxisAngle(left, pitch)); Vector3 yaw = inputclass.getyaw(); //horizontal swipe/stick forward = Vector3.transform(forward, Matrix.CreateFromAxisAngle(up, yaw)); left = Vector3.transform(left, Matrix.CreateFromAxisAngle(up, yaw)); forward.Normalize(); left.Normalize(); top.Normalize(); //avoid rounding errors world = Matrix.CreateWorld( postition //this isn't defined in my code forward, up); } } 

Since you have to have a view matrix to send to the shader, and your camera's rotation already in it, and it's an ortho-normal matrix, why not use it as your 'storage' device?

Assuming whatever framework you are using has methods for transposing matrices, rotations about arbitrary axis, multiplying a vector by a matrix(transforming), & building a matrix from 2 vectors. Here's some xna code but can be thought of as pseudo code.

Vector3 camPosition; Vector3 camTarget = new Vector3(0, 0, -1);//init to whatever you want. does not need to be unit length Matrix view = Matrix.Identity; //these items should be updated each frame before updating camera float pitchChangeFromLastFrame, yawChangeFromLastFrame;//long names just to clarify meaning float arcPitchChangeFromLastFrame, arcYawChangeFromLastFrame; Vector3 pointToArcAbout;//usually the camTarget Vector3 strafe; public void CameraUpdate() { Matrix camWorld = Matrix.Transpose(view); //pitch camera (tilt) camTarget = Vector3.Transform(camTarget - camPosition, Matrix.CreateFromAxisAngle(camWorld.Right, pitchChangeFromLastFrame)) + camPosition; //yaw camera (pan) camTarget = Vector3.Transform(camTarget - camPosition, Matrix.CreateFromAxisAngle(camWorld.Up, yawChangeFromLastFrame)) + camPosition;//often you would use 0,1,0 instead of camWorld.Up //strafe camera (track) camPosition += strafe; camTarget += strafe; //arc camera (pitch) camPosition = Vector3.Transform(camPosition - pointToArcAbout, Matrix.CreateFromAxisAngle(camWorld.Right, arcPitchChangeFromLastFrame)) + pointToArcAbout; //arc camera (yaw) camPosition = Vector3.Transform(camPosition - pointToArcAbout, Matrix.CreateFromAxisAngle(camWorld.Up, arcYawChangeFromLastFrame)) + pointToArcAbout; //rebuild view matrix view = Matrix.CreateLookAt(camPosition, camTarget, Vector3.Up);//creates an ortho normal matrix each frame from 2 vectors (camTarget - camPosition, & the up vector) } 

Essentially this takes 2 points in space (position & target) and rotates them about each other (using local axes) or moves them together (strafe), then builds a brand new ortho normal matrix each frame from them.