Pablo is absolutely on the right track. What is needed is a projection model that takes the position of the camera into account, as well as the lens characteristics. That model would be able to handle correctly all compositing problems that do not depend on the structure of the scene -- and is a prerequisite for solving the ones that do, i.e. parallax.
I think we should take this opportunity to put a complete 3D camera model into libpano and Hugin, as a basis for developing solutions to all the various kinds of "moving camera" problems panographers face.
There are many examples of the use of such models in the fields of photogrammetry and machine vision, and plenty of published discussions. So this is not a theoretical problem, but an engineering one.
The main problem posed by this change would be that a single panosphere is no longer the canonical reference surface. Instead there is such a sphere for each camera position, and all those spheres must be mapped to a compositing surface that may have to be custom-designed for a given image. The single panosphere is implicitly assumed at many places in libpano (and presumably Hugin too) so it could be a lot of work to implement this generalization. But very well worth the effort, in my opinion.
D M German wrote:Thank you for the pdf, it explains the idea.
> Pablo dAngelo twisted the bytes to say:
> Pablo> I'm still struggling to understand the geometric interpretation of the Tx,Ty,Tz angles and the Ts parameters.
> hi Pablo,
> I have created this document to explain the use of the three parameters.
For interested people on hugin-ptx:
Read this to see the context of the discussion
The pdf mentioned can be downloaded from
No, actually the rotation happens in image coordinates, not in its own
> The purpose of the tilt is to remap a photo to match another plane.
> basically, the second image is moved in an angle in its own sphere (Tx,
> Ty, and Tz, similar to yaw, pitch and roll)
sphere. This is a fundamental difference (in my opinion, it is also a
fundamental flaw). At least thats how it is implemented.
Thus, I don't see how the currently implemented tilt parameters allow a
general modelling, even when using only rectilinear -> rectilinear.
The XYZ mode I recently implemented actually does its rotation in an own
It rotates image B in its own sphere and then translates (X,Y) and
scales (Z) the input image (still on the sphere!), which is then
projected onto the panorama plane (when using rectilinear output). When
using another output projection than rectilinear, the image is projected
onto a intermediate plane, from where it is reprojected into the output
In that repect, I think the XYZ mode actually fits you description
better than the tilt mode.
Note that for fisheye images, it does not make sense to imagine image A
and image B as a plane, as it is illustrated in your example.
For that case, the transformation (ignoring lens distortion etc.) is a
simple matrix multiplication of the pixel coordinates with a homography
matrix T (T is a 3x3 matrix, with T(3,3) = 1). So that leaves 8 free
parameters that need to be determined (note that this also allows
non-square pixels, so effectively 7 parameters are needed for most
With the tilt mode, at least
v,r,p,y, Tx,Ty,Ts and d,e are required for the warping, so there must be
something wrong the model, as it should only require 7 parameters and
not 9. Using an overparametrized model is not good for the optimisation,
as it will confuse the optimizer.
With the XYZ model, only v,r,p,y, Tx,Ty,Tz are required, which is the
minimum amount of parameters, and they can handle a all possible camera
positions. This should work also work independently of the projection of
the input images.
Come build with us! The BlackBerry® Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9-12, 2009. Register now!
PanoTools-devel mailing list