1
$\begingroup$

Suppose, I've a 3d world point $P$ and I want to project it onto the image plane of camera, I can do it as follows $$M = \begin{bmatrix} R & t \\ 0 & 1 \\ \end{bmatrix}$$

$$p = K \cdot M \cdot P$$ where $K$ is the camera intrinsic matrix and $p$ gives my image point in homogeneous coordinates.

Now, I want to do the same thing for VR images (360° or 180° degree). Can you provide me any links for the math involved here?

I've attached two sample images here where the camera moves forward. As you can see the change in the frames is not the same when camera moved forward in normal videos. I believe it uses some other coordinate system (may be spherical) and also the images are such that when viewed closely on a headset, it gives 3d feeling.

frame1 frame2

Improve this question
$\endgroup$
2
  • $\begingroup$ paulbourke.net/miscellaneous/sphere2pano $\endgroup$ Commented Jul 15, 2020 at 13:35
  • $\begingroup$ Your images don't look like being projected onto a plane, but on a vertical cylinder. Central-projection math still applies, but the projection surface is curved. $\endgroup$ Commented Jul 15, 2020 at 14:10

1 Answer 1

Reset to default
3
$\begingroup$

Your images look like being shot with an angular fisheye projection . Also, to account for epipolar geometry of stereo images, you should include the fundamental matrix into the transform engine of your geometry pipeline.

To render a 3D scene in OpenCV with a fisheye camera, one uses a fisheye camera model. Regardless of a graphics software you are using, you may find useful a description of this model's math in an OpenCV documentation.

EDIT:

When using an application like Blender, you need not construct matrices and call API methods. You just select values for dialog controls. For example, to select a fisheye lens, you do it in accord with the following ref.: https://docs.blender.org/manual/en/2.79/render/cycles/camera.html :

CAMERA

. Lens

. Type : Fisheye

See also https://www.veer.tv/blog/blender-tutorial-how-to-render-a-3d-vr-video-from-blender/

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you so much for the references to start with. I've the blend file that generated the images in question. Do you know where I can check in blender the above information? $\endgroup$ Commented Jul 15, 2020 at 11:13
  • 1
    $\begingroup$ docs.blender.org/manual/en/2.79/render/cycles/camera.html : CAMERA Lens Type : Fisheye $\endgroup$ Commented Jul 15, 2020 at 11:52
  • $\begingroup$ Thanks. It is Panoramic Equirectangular in mine. I found it under Properties > Object Data Properties > Camera > Lens > Type. $\endgroup$ Commented Jul 15, 2020 at 12:40

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.