## panotools-devel

 [PanoTools-devel] tilt functions From: Pablo dAngelo - 2009-09-22 11:46:51 ```Hi all, I'm still struggling to understand the geometric interpretation of the Tx,Ty,Tz angles and the Ts parameters. I tried reading the source, but I'm now a little confused: In SetMakeParam() the tiltInverse is added after the projection into the input image. That is, the tiltInverse gets the image coordinates in the input image and transforms them into other coordinates in the input image. This seems strange, as the tile parameters are then dependent on the projection of the input files. I thought they somehow contain information where the camera is located with respect to the "master panorama camera". This also means the they are hard to interpret as they do not have a physical meaning. ciao Pablo ________________________________________________________________ Neu: WEB.DE Doppel-FLAT mit Internet-Flatrate + Telefon-Flatrate für nur 19,99 Euro/mtl.!* http://produkte.web.de/go/02/ ```
 Re: [PanoTools-devel] tilt functions From: Thomas Sharpless - 2009-09-22 14:07:16 Attachments: Message as HTML ```Hi All I too find this tilting method somewhat awkward. It seems the tilt is done by rotating around panocenter, which makes it necessary to translate the tile center back to where it belongs. If the rotation were around the tile center instead, then no translation would be needed. It is just a matter of setting the Z coordinate to 0 before rotating. Regards, Tom On Tue, Sep 22, 2009 at 7:19 AM, Pablo dAngelo wrote: > Hi all, > > I'm still struggling to understand the geometric interpretation of the > Tx,Ty,Tz angles and the Ts parameters. > > I tried reading the source, but I'm now a little confused: > > In SetMakeParam() the tiltInverse is added after the projection into the > input image. That is, > the tiltInverse gets the image coordinates in the input image and > transforms them into other > coordinates in the input image. > This seems strange, as the tile parameters are then dependent on the > projection of the input files. I thought they somehow contain information > where the camera is located with respect to the "master panorama camera". > This also means the they are hard to interpret as they do not have a > physical > meaning. > > ciao > Pablo > > > ________________________________________________________________ > Neu: WEB.DE Doppel-FLAT mit Internet-Flatrate + Telefon-Flatrate > für nur 19,99 Euro/mtl.!* http://produkte.web.de/go/02/ > > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > PanoTools-devel mailing list > PanoTools-devel@... > https://lists.sourceforge.net/lists/listinfo/panotools-devel > ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-22 18:17:14 ```Hi all, After some more playing around with the tilt functions, I decided to scrap them and implement a physically meaningful model (Using the approach I recommended to Dev at LGM, Jim might remember it). A few hours later, and I now have a working forward and inverse model that can be used to estimate the camera position in X,Y and Z. This allows a nice, physically consistent modelling that does not misuse other parameter (shift etc.), that are usually required for other purposes. It should also be more stable the the tilt model, as it is not parametrized with angles and doesn't suffer from singularities (Ts=0 and problems for large Tx and Ty angles with the tilt model). It was actually pretty easy to implement. I'm currently misusing the tilt parameter Tx,Ty,Tz for the camera position, but I should probably use the unused X,Y,Z parameters instead, but I haven't check if they are available in the SetMakeParam() functions (yet). I'd like to commit my version and remove the tilt model, as it doesn't fit into the panotools modelling approach. Is that ok? ciao Pablo Thomas Sharpless schrieb: > Hi All > > I too find this tilting method somewhat awkward. It seems the tilt is > done by rotating around panocenter, which makes it necessary to > translate the tile center back to where it belongs. > > If the rotation were around the tile center instead, then no translation > would be needed. It is just a matter of setting the Z coordinate to 0 > before rotating. > > Regards, Tom > > On Tue, Sep 22, 2009 at 7:19 AM, Pablo dAngelo > wrote: > > Hi all, > > I'm still struggling to understand the geometric interpretation of > the Tx,Ty,Tz angles and the Ts parameters. > > I tried reading the source, but I'm now a little confused: > > In SetMakeParam() the tiltInverse is added after the projection into > the input image. That is, > the tiltInverse gets the image coordinates in the input image and > transforms them into other > coordinates in the input image. > This seems strange, as the tile parameters are then dependent on the > projection of the input files. I thought they somehow contain > information > where the camera is located with respect to the "master panorama > camera". > This also means the they are hard to interpret as they do not have a > physical > meaning. > > ciao > Pablo > > > ________________________________________________________________ > Neu: WEB.DE ; Doppel-FLAT mit Internet-Flatrate + > Telefon-Flatrate > für nur 19,99 Euro/mtl.!* http://produkte.web.de/go/02/ > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and > stay > ahead of the curve. Join us from November 9-12, 2009. Register > now! > http://p.sf.net/sfu/devconf > _______________________________________________ > PanoTools-devel mailing list > PanoTools-devel@... > > https://lists.sourceforge.net/lists/listinfo/panotools-devel > > > > ------------------------------------------------------------------------ > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > > > ------------------------------------------------------------------------ > > _______________________________________________ > PanoTools-devel mailing list > PanoTools-devel@... > https://lists.sourceforge.net/lists/listinfo/panotools-devel ```
 Re: [PanoTools-devel] tilt functions From: D M German - 2009-09-22 19:10:52 ``` Pablo> After some more playing around with the tilt functions, I decided to Pablo> scrap them and implement a physically meaningful model (Using the Pablo> approach I recommended to Dev at LGM, Jim might remember it). Pablo> A few hours later, and I now have a working forward and inverse model Pablo> that can be used to estimate the camera position in X,Y and Z. This Pablo> allows a nice, physically consistent modelling that does not misuse Pablo> other parameter (shift etc.), that are usually required for other purposes. Hi Pablo, I think it is great that you implemented this. But, are both methods solving the same problem? I think they are not. Imagine a very long wall, all of which is a single plane. The tilt model does not assume a translation of the center of the image, only a remapping of the plane. In such a case, as the person walks along the wall, the angle for tilt is small, and the scale close to 1. If I understand correctly, this new positional model moves the photo viewing point to the center of the panosphere. As the wall gets longer, I would imagine the error becomes larger. and we have the problem of the sphere being bound at 360 degrees. Correct me if I am wrong, but I think the tilt functions can be used for mosaics of a known plane better than the spherical model. Please commit the changes. I'll add new parameters for your model in the parser. Just tell me what you need. --dmg -- -- Daniel M. German http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with . ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-22 20:14:54 ```D M German wrote: > > Hi Pablo, > > I think it is great that you implemented this. > > But, are both methods solving the same problem? I think they are not. > > Imagine a very long wall, all of which is a single plane. The tilt model > does not assume a translation of the center of the image, only a > remapping of the plane. In such a case, as the person walks along the > wall, the angle for tilt is small, and the scale close to 1. > > If I understand correctly, this new positional model moves the photo > viewing point to the center of the panosphere. As the wall gets longer, > I would imagine the error becomes larger. and we have the problem of the > sphere being bound at 360 degrees. Actually, the new model is bound to a plane, so it can't handle anything more than 179 degrees. If the pano is straightened properly, the rectilinear view of the plane won't be distorted, even if it is extremly wide. For an illustration, take a panorama that has been captured close to a wide (and completely flat) facade, and view it in an interative viewer. Zoom out to an extremely wide view (lets say hfov=130 or so). If you view the facade head on without distortion. If you move a little, it will start to exhibit lots of perspective "distortion". I think the fact that there is only one plane through which the images are remapped could actually improve stability for longer areas. In my current implementation, the plane on which the non center cameras are projected is currently fixed to z=-1 (straight ahead). The default pano camera is then located at 0,0,0 and we only images that have X,Y,Z != 0 go through this plane. This means that images that are located at the origin can be stitched to full spherical without problems. This is fine for normal mosaics, but for nadir patching, the plane should be in the at y=-1. Ideally the plane should be configurable. > Correct me if I am wrong, but I think the tilt functions can be used for > mosaics of a known plane better than the spherical model. I guess both could be used. I tried with the aerial images posted on hugin-ptx and I didn't manage to get a decent result with even only a small number of images. With my new model is is much better but not perfect yet. Once both things are in, we can try and see which the pros and cons of each method. > Please commit the changes. I'll add new parameters for your model in the > parser. Just tell me what you need. Ok, I have just commited my changes. If you open adjust.c and search for MOSAIC_XYZ, you can see what is required. If MOSAIC_XYZ is defined, the tilt mode is replaced with the method described above. I'd like to reuse the X,Y,Z parameters in the script, but I'd have to do quite some digging to introduce them to the optimizer, you probably know better which places need to be modified. It would be great if you could add these to the correct_Prefs struct and the places where the optimizer needs it. ciao Pablo ```
 Re: [PanoTools-devel] tilt functions From: D M German - 2009-09-22 20:50:04 ``` Pablo d'Angelo twisted the bytes to say: >> Please commit the changes. I'll add new parameters for your model in the >> parser. Just tell me what you need. Pablo> Ok, I have just commited my changes. If you open adjust.c and search for Pablo> MOSAIC_XYZ, you can see what is required. If MOSAIC_XYZ is defined, the Pablo> tilt mode is replaced with the method described above. Pablo> I'd like to reuse the X,Y,Z parameters in the script, but I'd have to do Pablo> quite some digging to introduce them to the optimizer, you probably know Pablo> better which places need to be modified. It would be great if you could Pablo> add these to the correct_Prefs struct and the places where the optimizer Do you mean creating three parameters X, Y, Z in the parser? How about we create Mx, My, and Mz? We need to be worry about running out of letters :) I can do that, not a problem. --dmg -- -- Daniel M. German http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with . ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-22 21:00:03 ```D M German wrote: > Pablo d'Angelo twisted the bytes to say: > > Pablo> I'd like to reuse the X,Y,Z parameters in the script, but I'd have to do > Pablo> quite some digging to introduce them to the optimizer, you probably know > Pablo> better which places need to be modified. It would be great if you could > Pablo> add these to the correct_Prefs struct and the places where the optimizer > > Do you mean creating three parameters X, Y, Z in the parser? The parser already knows about X,Y,Z, but it is not stored in the correct_Prefs structure, but in some ImageCoords struct, which is not handled by the panorama part of pano13. > How about we create Mx, My, and Mz? We need to be worry about running > out of letters :) I'd introduce new variable only if there is a problem with reusing X,Y,Z. I don't see a problem with reusing X,Y,Z. They are already parsed, but only used by PTstereo (which can't use libpano13 either...). Even if some day, PTstereo would magically appear, it shouldn't be a problem, if a copy of the values are stored in the correct_Prefs structure. ciao Pablo ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-22 21:50:46 ```Pablo d'Angelo wrote: > D M German wrote: > >> Pablo d'Angelo twisted the bytes to say: >> >> Pablo> I'd like to reuse the X,Y,Z parameters in the script, but I'd have to do >> Pablo> quite some digging to introduce them to the optimizer, you probably know >> Pablo> better which places need to be modified. It would be great if you could >> Pablo> add these to the correct_Prefs struct and the places where the optimizer >> >> Do you mean creating three parameters X, Y, Z in the parser? >> > > The parser already knows about X,Y,Z, but it is not stored in the > correct_Prefs structure, but in some ImageCoords struct, which is not > handled by the panorama part of pano13. > PTAInterpolate uses the X, Y, Z parameters There is a sample project linked here using PTInterpolate. http://wiki.panotools.org/PTInterpolate PTAInterpolate is an enhanced replacement version of PTInterpolate by Max Lyons and is in the tools folder. http://www.tawbaware.com/maxlyons/pano12ml.htm >> How about we create Mx, My, and Mz? We need to be worry about running >> out of letters :) >> > > I'd introduce new variable only if there is a problem with reusing > X,Y,Z. I don't see a problem with reusing X,Y,Z. They are already > parsed, but only used by PTstereo (which can't use libpano13 either...). > Even if some day, PTstereo would magically appear, it > shouldn't be a problem, if a copy of the values are stored in the > correct_Prefs structure. > > ciao > Pablo > Is it possible that PTAInterpolate using X, Y, and Z in the same way as your changes? I'll try looking at the code tonight. -- Jim Watters jwatters@... http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-23 06:07:09 ```The sample project for Interpolate and Stereo do not show the X, Y, & Z parameters being optimized. For the three image stereo project example, X has starting values of 0, 1, and 2 while Y & Z are all 0 for all three images. It does not look like X, Y, & Z are even used with PTAInterpolate. It will only interpolate 2 images while I imagine the original version used X to define the position (in morph space) of multiple images. Everything is in place to read, parse, and verify the X, Y, & Z parameters. The problem would be breaking Interpolate and Stereo if by optimizing XYZ to bad values and saving them to the script. Pablo, I think you are correct is saying that the impact of using X, Y, Z is small. It would be nice to use the documentation that already describes XYZ as the Real World Coordinates of the image. What are the chances that PTStereo and your code is using XYZ in the same way? I am not sure if we will need to add to correct_Prefs struct so it can be optimized. If so, a search on Tilt will reveal everywhere to update. If we do create new variables in the struct, The biggest problem will be when writing out the script after XYZ have been optimized. Which values do you write. Although, It will make the scripts neater. The code wont be any simpler because the parameters will need to be stored in two locations and writing out the script might cause issues. It might just be simpler to create a new set of parameters. Jim Jim Watters wrote: > Pablo d'Angelo wrote: > >> D M German wrote: >> >> >>> Pablo d'Angelo twisted the bytes to say: >>> >>> Pablo> I'd like to reuse the X,Y,Z parameters in the script, but I'd have to do >>> Pablo> quite some digging to introduce them to the optimizer, you probably know >>> Pablo> better which places need to be modified. It would be great if you could >>> Pablo> add these to the correct_Prefs struct and the places where the optimizer >>> >>> Do you mean creating three parameters X, Y, Z in the parser? >>> >>> >> The parser already knows about X,Y,Z, but it is not stored in the >> correct_Prefs structure, but in some ImageCoords struct, which is not >> handled by the panorama part of pano13. >> >> > PTAInterpolate uses the X, Y, Z parameters > There is a sample project linked here using PTInterpolate. > http://wiki.panotools.org/PTInterpolate > PTAInterpolate is an enhanced replacement version of PTInterpolate by > Max Lyons and is in the tools folder. > http://www.tawbaware.com/maxlyons/pano12ml.htm > > >>> How about we create Mx, My, and Mz? We need to be worry about running >>> out of letters :) >>> >>> >> I'd introduce new variable only if there is a problem with reusing >> X,Y,Z. I don't see a problem with reusing X,Y,Z. They are already >> parsed, but only used by PTstereo (which can't use libpano13 either...). >> Even if some day, PTstereo would magically appear, it >> shouldn't be a problem, if a copy of the values are stored in the >> correct_Prefs structure. >> >> ciao >> Pablo >> >> > Is it possible that PTAInterpolate using X, Y, and Z in the same way as > your changes? > I'll try looking at the code tonight. > > -- Jim Watters http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-24 18:55:00 ```Hi Jim, Jim Watters wrote: > The sample project for Interpolate and Stereo do not show the X, Y, & Z > parameters being optimized. > For the three image stereo project example, X has starting values of 0, > 1, and 2 while Y & Z are all 0 for all three images. > > It does not look like X, Y, & Z are even used with PTAInterpolate. It > will only interpolate 2 images while I imagine the original version used > X to define the position (in morph space) of multiple images. > > Everything is in place to read, parse, and verify the X, Y, & Z parameters. > The problem would be breaking Interpolate and Stereo if by optimizing > XYZ to bad values and saving them to the script. > > Pablo, I think you are correct is saying that the impact of using X, Y, > Z is small. It would be nice to use the documentation that already > describes XYZ as the Real World Coordinates of the image. What are the > chances that PTStereo and your code is using XYZ in the same way? As far as I can see by reading the documentation I use similar axes conventions, but my parameters probably cannot be used by PTStereo, as they are probably only compatible with PTStereo when all objects are located on a plane at z=-1. For this scenario using PTStereo wouldn't make much sense, however. Do you have any plans for implementing PTStereo? I'd rather work on improving libs like sba when I need real 3D bundle adjustment, than fighting with the libpano code. > If we do create new variables in the struct, The biggest problem will be > when writing out the script after XYZ have been optimized. Which values > do you write. Although, It will make the scripts neater. The code wont > be any simpler because the parameters will need to be stored in two > locations and writing out the script might cause issues. It might just > be simpler to create a new set of parameters. Hmm, I don't really care about the name in the file, so Mx, My, Mz would be fine with me, if it is too hard to reuse X,Y,Z. ciao Pablo ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-24 19:17:42 ```Pablo d'Angelo wrote: > Hi Jim, > > Jim Watters wrote: > >> Everything is in place to read, parse, and verify the X, Y, & Z parameters. >> The problem would be breaking Interpolate and Stereo if by optimizing >> XYZ to bad values and saving them to the script. >> > Do you have any plans for implementing PTStereo? I'd rather work on > improving libs like sba when I need real 3D bundle adjustment, than > fighting with the libpano code. > I have no plans on implementing PTStereo. The more I think about it, the more I want to modify the structures by moving the coordinate data. If someone does create PTStereo it can still be used from there. Then it also makes sense to remove the coordinate data in the current location. If it is left along it will only make reading and writing script in the future that much more confusing. If Daniel agrees with the changes, I'll do it by the end of the weekend if no one does it first. -- Jim Watters jwatters@... http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: D M German - 2009-09-22 20:52:46 ``` Pablo> I think the fact that there is only one plane through which the images Pablo> are remapped could actually improve stability for longer areas. Hi Pablo, Are you referring here to the Mosaic or the tilt mode? Or both? --dmg -- Daniel M. German http://turingmachine.org/ http://silvernegative.com/ dmg (at) uvic (dot) ca replace (at) with @ and (dot) with . ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-24 04:42:18 ```D M German schrieb: > > Pablo> I think the fact that there is only one plane through which the images > Pablo> are remapped could actually improve stability for longer areas. > > Hi Pablo, > > Are you referring here to the Mosaic or the tilt mode? Or both? I'm refering the the mosaic (XYZ) mode, as the tilt mode has no concept of a global plane (Its just a bunch of parameters to wrap each image pair separately onto each other). This is why I think the XYZ mode should be more stable. However I have also reached its limitations when playing around with the aerial imagery posted in hugin-ptx (they are pretty extreme), as there are problems when the yaw angle becomes close to 90° during the optimization. Then it can't be projected through the plane anymore, as the camera starts to face backwards. So for complicated scenes, a careful optimisation is also needed for the XYZ mode. ciao Pablo ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-24 19:29:34 ```Pablo d'Angelo wrote: > D M German schrieb: > >> Pablo> I think the fact that there is only one plane through which the images >> Pablo> are remapped could actually improve stability for longer areas. >> >> Hi Pablo, >> >> Are you referring here to the Mosaic or the tilt mode? Or both? >> > > I'm refering the the mosaic (XYZ) mode, as the tilt mode has no concept > of a global plane (Its just a bunch of parameters to wrap each image > pair separately onto each other). This is why I think the XYZ mode > should be more stable. > > However I have also reached its limitations when playing around with the > aerial imagery posted in hugin-ptx (they are pretty extreme), as there > are problems when the yaw angle becomes close to 90° during the > optimization. Then it can't be projected through the plane anymore, as > the camera starts to face backwards. So for complicated scenes, a > careful optimisation is also needed for the XYZ mode. > > ciao > Pablo > > Maybe the code should be forked to create a libmosaic that does not force the idea that the images create a single sphere and taken from a single NPP (or viewed from a single point)? Or maybe we need another output projection? -- Jim Watters jwatters@... http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-25 02:31:16 ```Pablo d'Angelo wrote: > I'm refering the the mosaic (XYZ) mode, as the tilt mode has no concept > of a global plane (Its just a bunch of parameters to wrap each image > pair separately onto each other). This is why I think the XYZ mode > should be more stable. > > However I have also reached its limitations when playing around with the > aerial imagery posted in hugin-ptx (they are pretty extreme), as there > are problems when the yaw angle becomes close to 90° during the > optimization. Then it can't be projected through the plane anymore, as > the camera starts to face backwards. So for complicated scenes, a > careful optimisation is also needed for the XYZ mode. > > ciao > Pablo > Is the limit caused by output projection being rectilinear? would an orthophoto projection work better, if we can optimize for that? Is there a limit to how far the camera can moves along an axis get further away from the origin? -- Jim Watters http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: D M German - 2009-09-22 18:58:14 Attachments: text/plain panotoolsTilt.pdf text/plain
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-24 05:44:55 ```D M German wrote: > Pablo dAngelo twisted the bytes to say: > > Pablo> I'm still struggling to understand the geometric interpretation of the Tx,Ty,Tz angles and the Ts parameters. > > hi Pablo, > > I have created this document to explain the use of the three parameters. Thank you for the pdf, it explains the idea. For interested people on hugin-ptx: Read this to see the context of the discussion http://thread.gmane.org/gmane.comp.graphics.panotools.devel/1517 The pdf mentioned can be downloaded from http://cache.gmane.org//gmane/comp/graphics/panotools/devel/1522-001.bin > The purpose of the tilt is to remap a photo to match another plane. > > basically, the second image is moved in an angle in its own sphere (Tx, > Ty, and Tz, similar to yaw, pitch and roll) No, actually the rotation happens in image coordinates, not in its own sphere. This is a fundamental difference (in my opinion, it is also a fundamental flaw). At least thats how it is implemented. Thus, I don't see how the currently implemented tilt parameters allow a general modelling, even when using only rectilinear -> rectilinear. The XYZ mode I recently implemented actually does its rotation in an own sphere: It rotates image B in its own sphere and then translates (X,Y) and scales (Z) the input image (still on the sphere!), which is then projected onto the panorama plane (when using rectilinear output). When using another output projection than rectilinear, the image is projected onto a intermediate plane, from where it is reprojected into the output projection. In that repect, I think the XYZ mode actually fits you description better than the tilt mode. Note that for fisheye images, it does not make sense to imagine image A and image B as a plane, as it is illustrated in your example. For that case, the transformation (ignoring lens distortion etc.) is a simple matrix multiplication of the pixel coordinates with a homography matrix T (T is a 3x3 matrix, with T(3,3) = 1). So that leaves 8 free parameters that need to be determined (note that this also allows non-square pixels, so effectively 7 parameters are needed for most usecases). With the tilt mode, at least v,r,p,y, Tx,Ty,Ts and d,e are required for the warping, so there must be something wrong the model, as it should only require 7 parameters and not 9. Using an overparametrized model is not good for the optimisation, as it will confuse the optimizer. With the XYZ model, only v,r,p,y, Tx,Ty,Tz are required, which is the minimum amount of parameters, and they can handle a all possible camera positions. This should work also work independently of the projection of the input images. ciao Pablo ```
 Re: [PanoTools-devel] tilt functions From: Thomas Sharpless - 2009-09-24 14:29:25 Attachments: Message as HTML ```Hi All Pablo is absolutely on the right track. What is needed is a projection model that takes the position of the camera into account, as well as the lens characteristics. That model would be able to handle correctly all compositing problems that do not depend on the structure of the scene -- and is a prerequisite for solving the ones that do, i.e. parallax. I think we should take this opportunity to put a complete 3D camera model into libpano and Hugin, as a basis for developing solutions to all the various kinds of "moving camera" problems panographers face. There are many examples of the use of such models in the fields of photogrammetry and machine vision, and plenty of published discussions. So this is not a theoretical problem, but an engineering one. The main problem posed by this change would be that a single panosphere is no longer the canonical reference surface. Instead there is such a sphere for each camera position, and all those spheres must be mapped to a compositing surface that may have to be custom-designed for a given image. The single panosphere is implicitly assumed at many places in libpano (and presumably Hugin too) so it could be a lot of work to implement this generalization. But very well worth the effort, in my opinion. Best, Tom On Thu, Sep 24, 2009 at 1:44 AM, Pablo d'Angelo wrote: > D M German wrote: > > Pablo dAngelo twisted the bytes to say: > > > > Pablo> I'm still struggling to understand the geometric interpretation > of the Tx,Ty,Tz angles and the Ts parameters. > > > > hi Pablo, > > > > I have created this document to explain the use of the three parameters. > > Thank you for the pdf, it explains the idea. > > For interested people on hugin-ptx: > Read this to see the context of the discussion > http://thread.gmane.org/gmane.comp.graphics.panotools.devel/1517 > The pdf mentioned can be downloaded from > http://cache.gmane.org//gmane/comp/graphics/panotools/devel/1522-001.bin > > > The purpose of the tilt is to remap a photo to match another plane. > > > > basically, the second image is moved in an angle in its own sphere (Tx, > > Ty, and Tz, similar to yaw, pitch and roll) > > No, actually the rotation happens in image coordinates, not in its own > sphere. This is a fundamental difference (in my opinion, it is also a > fundamental flaw). At least thats how it is implemented. > > Thus, I don't see how the currently implemented tilt parameters allow a > general modelling, even when using only rectilinear -> rectilinear. > > The XYZ mode I recently implemented actually does its rotation in an own > sphere: > It rotates image B in its own sphere and then translates (X,Y) and > scales (Z) the input image (still on the sphere!), which is then > projected onto the panorama plane (when using rectilinear output). When > using another output projection than rectilinear, the image is projected > onto a intermediate plane, from where it is reprojected into the output > projection. > > In that repect, I think the XYZ mode actually fits you description > better than the tilt mode. > > Note that for fisheye images, it does not make sense to imagine image A > and image B as a plane, as it is illustrated in your example. > > For that case, the transformation (ignoring lens distortion etc.) is a > simple matrix multiplication of the pixel coordinates with a homography > matrix T (T is a 3x3 matrix, with T(3,3) = 1). So that leaves 8 free > parameters that need to be determined (note that this also allows > non-square pixels, so effectively 7 parameters are needed for most > usecases). > With the tilt mode, at least > v,r,p,y, Tx,Ty,Ts and d,e are required for the warping, so there must be > something wrong the model, as it should only require 7 parameters and > not 9. Using an overparametrized model is not good for the optimisation, > as it will confuse the optimizer. > > With the XYZ model, only v,r,p,y, Tx,Ty,Tz are required, which is the > minimum amount of parameters, and they can handle a all possible camera > positions. This should work also work independently of the projection of > the input images. > > ciao > Pablo > > > ------------------------------------------------------------------------------ > Come build with us! The BlackBerry® Developer Conference in SF, CA > is the only developer event you need to attend this year. Jumpstart your > developing skills, take BlackBerry mobile applications to market and stay > ahead of the curve. Join us from November 9-12, 2009. Register now! > http://p.sf.net/sfu/devconf > _______________________________________________ > PanoTools-devel mailing list > PanoTools-devel@... > https://lists.sourceforge.net/lists/listinfo/panotools-devel > ```
 Re: [PanoTools-devel] tilt functions From: Jim Watters - 2009-09-24 14:26:52 ```Pablo d'Angelo wrote: > D M German wrote: > >> Pablo dAngelo twisted the bytes to say: >> >> Pablo> I'm still struggling to understand the geometric interpretation of the Tx,Ty,Tz angles and the Ts parameters. >> >> hi Pablo, >> >> I have created this document to explain the use of the three parameters. >> > > Thank you for the pdf, it explains the idea. > > For interested people on hugin-ptx: > Read this to see the context of the discussion > http://thread.gmane.org/gmane.comp.graphics.panotools.devel/1517 > The pdf mentioned can be downloaded from > http://cache.gmane.org//gmane/comp/graphics/panotools/devel/1522-001.bin > > >> The purpose of the tilt is to remap a photo to match another plane. >> >> basically, the second image is moved in an angle in its own sphere (Tx, >> Ty, and Tz, similar to yaw, pitch and roll) >> > > No, actually the rotation happens in image coordinates, not in its own > sphere. This is a fundamental difference (in my opinion, it is also a > fundamental flaw). At least thats how it is implemented. > > Thus, I don't see how the currently implemented tilt parameters allow a > general modelling, even when using only rectilinear -> rectilinear. > > The XYZ mode I recently implemented actually does its rotation in an own > sphere: > It rotates image B in its own sphere and then translates (X,Y) and > scales (Z) the input image (still on the sphere!), which is then > projected onto the panorama plane (when using rectilinear output). When > using another output projection than rectilinear, the image is projected > onto a intermediate plane, from where it is reprojected into the output > projection. > > In that repect, I think the XYZ mode actually fits you description > better than the tilt mode. > > Note that for fisheye images, it does not make sense to imagine image A > and image B as a plane, as it is illustrated in your example. > > For that case, the transformation (ignoring lens distortion etc.) is a > simple matrix multiplication of the pixel coordinates with a homography > matrix T (T is a 3x3 matrix, with T(3,3) = 1). So that leaves 8 free > parameters that need to be determined (note that this also allows > non-square pixels, so effectively 7 parameters are needed for most > usecases). > With the tilt mode, at least > v,r,p,y, Tx,Ty,Ts and d,e are required for the warping, so there must be > something wrong the model, as it should only require 7 parameters and > not 9. Using an overparametrized model is not good for the optimisation, > as it will confuse the optimizer. > > With the XYZ model, only v,r,p,y, Tx,Ty,Tz are required, which is the > minimum amount of parameters, and they can handle a all possible camera > positions. This should work also work independently of the projection of > the input images. > > ciao > Pablo > If I understand correctly both models will align an image so it is on the same plane as others. The XYZ model calculates the position of the camera, this will cause certain distortions to the image. XYZ have to be optimized with ypr. The TxTyTzTs model calculates the actual distortion to align the image after it has been placed correctly after optimizing ypr. The advantage to the XYZ model is it uses fewer overall parameters. But when I try imagine manipulating images in my head using TxTyTzTs i don't see how Tz is any different than r. And maybe Tz it is not necessary. The advantage to the TxTyTs model is ypr can be optimized first to get a good placement before distorting the image by optimizing TxTyTs overall needing fewer control points. How wrong am I? Before the weekend is done I will create some test cases to compare the two. -- Jim Watters jwatters@... http://photocreations.ca ```
 Re: [PanoTools-devel] tilt functions From: Pablo d'Angelo - 2009-09-24 18:39:38 ```Jim Watters schrieb: > Pablo d'Angelo wrote: >> > If I understand correctly both models will align an image so it is on > the same plane as others. After some more reading of the code, I'm quite sure that the tilt can only work well with rectilinear input images. If only that case is interesting, one doesn't even need angles, a simple 3x3 homography matrix transform is sufficient, and probably more stable than the tilt model. > The XYZ model calculates the position of the camera, this will cause > certain distortions to the image. XYZ have to be optimized with ypr. > The TxTyTzTs model calculates the actual distortion to align the image > after it has been placed correctly after optimizing ypr. I don't believe that ypr and the TxTyTzTs are independent. For the shift compensation, the TxTyTzTs model will also require simultanious optimisation of d and e. > The advantage to the XYZ model is it uses fewer overall parameters. > > But when I try imagine manipulating images in my head using TxTyTzTs i > don't see how Tz is any different than r. And maybe Tz it is not > necessary. The advantage to the TxTyTs model is ypr can be optimized > first to get a good placement before distorting the image by optimizing > TxTyTs overall needing fewer control points. You still need Tx, Ty, Ts and d,e. This assumes that you do not need to touch y,p,r from a previous optimisation. However, I think y,p,r will try to compensate for the parallax, and they will thus be quite wrong, in an attempt to rescue the situation. I'm not sure that a later tilt estimation works satisfactory with the previously estimated y,p,r parameters. Maybe one can get away, if the camera hasn't been moved too far. My experience with the tilt model was not so good. I have prepared a nice testset with the aerial images posted earlier, and it works quite well with the XYZ mode. I'll give more details later. ciao Pablo ```