To change the zoom without changing the image, modify your field of view angle. This effectively scales the image up or down, without changing the geometric relationships in the image plane — if two lines cross at a particular point, they'll continue crossing at that same point, with no parallax effects that move the apparent intersection relative to other image features.
To translate side to side or up and down (truck and boom/jib movements — perpendicular to the view direction), you can use an oblique camera frustum. This effectively shifts the image after the projection has been applied, so again you'll see no parallax. The effect can be strange though, if you use a large shift: you'll see an image projected from a viewpoint distant from the location in the center of the image, and you'll see more dramatic perspective distortion the farther you go from the camera's axis.
Taking the classic OpenGL projection matrix:
$$\left[\begin{array}{cccc} { \dfrac{2n}{ r-l } } & 0 & { \dfrac{r + l} { r-l } + x_{shift}} & 0 \\ 0 & { \dfrac{2n}{ t-b } } & { \dfrac{t + b}{ t-b } + y_{shift}} & 0 \\ 0 & 0 & -{\dfrac{f+n}{f-n}} & -{\dfrac{2fn}{f-n}}\\ 0 & 0 & -1& 0\\ \end{array}\right]$$
The \$x_{shift}\$ and \$y_{shift}\$ variables there control the oblique frustum. When they're zero, you get the usual projection, with the camera axis passing through the center of the image. As you adjust these values, you'll slide the image side to side without altering the perspective.
