From: Pablo d'Angelo <pablo.dangelo@we...>  20070317 21:16:40

Ippei UKAI schrieb: > On 20070316, at 03:11, Daniel M. German wrote: > >> Hi Ippei, >> >> I like the computation stack model. It is powerful. But it is also >> very difficult to improve and maintain. > > So it computes the composition of all the operations on the images > and apply it over the bitmap matrix. That's clever way to handle non > discrete intermediate coordinates. And this is one of the big strengths of panotools. We need to keep that. The transform stack can be treated as a black box, and can represent a all different kinds of 2D<>2D transformations. Currently panotools builds the stack so that everything goes through a spherical model internally, but this is not a fundamental requirement! Interesting features are possible if we allow other transformations as well. Just think how complicated this conceptually very elegant procedure is: http://wiki.panotools.org/Fixing_nadir_parallax_errors I believe the maintainance difficulties are more due to technical reasons and not due to the flexibility of the transform stack approach. > Sounds like we need something like > OpenGL Shading Language that can be manipulated on runtime and > executed very fast according to the output resolution. I have no idea > how it could be done in C to be honest though. I'm too used to OOP. Using hardware acceleration is an attractive idea, however generality should not be sacrified. I think it might be useful to build dedictated transformations for the most common use cases. >> Ippei> What I understood from above is that the >> Ippei> optimisation and output projection are independent of each >> other. >> >> No, it is just that when we added new projections we did not bother to >> add them as input projections, only as oputput ones. > > Okay, I think I'm still lost here. Why do we need the input versions > of the output projections for optimisation if the optimisation is > independent from the output projection process? Actually an output projection has to be specified for the optimisation, however, currently only rectilinear, cylindrical and spherical are available, with spherical being the logical solution for 360x180 deg panos. But lets step back. I will try to summarize how the stuff related to projections works in panotools. 1. remapping is quite simple: (pano image coordinate P_j) > [transform_stack_i(x_i)] > (image i coordinates I_ij) transform_stack_i(x_i) is the transformation function parametrized with different parameters x_i (yaw, roll, pitch, hfov) for each input image i. You can use different projections for each input image, thus different transform stacks are required. The pixel found in the image is then placed in the panorama image. Actually, in the literature this is usually called inverse mapping, since the forward mapping, which transforms the input image coordinates I into the panorama coordinates P has some drawbacks when actual images should be remapped: http://blogs.mathworks.com/steve/?p=53 2. In order to calculate the function above, the parameters x_i need to be determined. This is done using the optimizer. Here stuff gets tricky. There are two ways to attack the problem, and the second one I describe is used in panotools, so feel free to skip to II) if you head aches while reading I): I) using the same transformation as for the remapping: Assume that we roughly know the coordinate P_j in the panorama image and have measured the associated image coordinates I_ij (corresponding points in the images). Then we can transform P_j into each image, resulting in image coordinates I*_ij. In order to make the images match, we require I_ij = I*_ij With this equation, we can then estimate the coordinate of P_j, and the transformation parameters x_i. The advantage of this method is that it allows "multi image" control points, and the main drawbacks are that the coordinates P_j also need to be determined, making the problem harder to solve. Additionally approximate initial values for P_j need to be available (but could be computed using other methods). The idea behind this method is also used in Bundle Adjustment, as used extensively in Photogrammetry for 3D reconstruction and map building. II) using the inverse transformation. Contrary to 3D reconstruction from images, where the transformation is actually a projection from 3D>2D, and thus cannot be uniquely invertable, we have the advantage that our transformation is invertible Thus we can use another approach based on the inverse transformation: Use the control point I_1j and I_2j to calculate P_1j and P_2j. To make the images match, we require P_1j = P_2j The parameters x_j can then be recovered by minimising the distance between P_1j  P_2j, which is the distance of control points in the final panorama. As implemented in panotools, this distance is always measured on the sphere, except for the special horizontal, vertical and line control points. The big advantege is that we do not need to estimate values for P_j, and thus have fewer parameters to estimate, and fewer control points are required. There are also other differences with respect to stability etc. This is why the inverse transformations are required for the optimisation. I hope this was not to confusing. ciao Pablo 