[1442bb]: RevisionLog.txt Maximize Restore History

Download this file

RevisionLog.txt    324 lines (277 with data), 14.0 kB

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
APSCpp Revision Log started 18 July 2009 TKSharpless
18 July some quick & easy changes
-- #define PACKAGE_VERSION "2.5.1" in AutoPanoSift.h
TODO: separate APSCpp version no. from gk/ap one
-- correct code for smoothing image before size reduction
1) delete orignal image and swap the pointers
2) do it in the if(Scale > 1) block
-- add user controls on stereographic projection
1) --stereographic off prevents conversion
2) --stgfov sets minimum hfov for conversion. Show this
in the setup report on stereographic option
implement as new argument stgfov to GenerateKeyspp()
(< 0 prohibits conversion)
-- remove alignment and related options from APSCpp_main.c
1) delete from usage text
2) delete corresponding commandline parsing
3) delete all code that referred to bondball
4) pass null alignment params to write_pto (as if the
option had been switched off)
TODO: remove references in other files,verify that linker
nolonger loads bondball from libsift.
-- add (dummy) option --align-aware to control proposed
layout-based keypoint selection.
-- revise the usage text and some other explanatory messages
for better clarity.
-- Display revision number in startup banner.
-- Add a couple of status reports to better delineate image
processing stages.
-- verify no cumulative memory loss during keypoint finding.
-- add this file to svn, commit as SVN 4054
19 July Further Q&D's
-- change "align-aware" to "use-layout" (clearer)
-- eliminate option to use legacy k-d tree, ANN only.
1) remove usage line and command parse
2) remove control var useANN & uses of it
3) eliminate mode argument to GenerateKeyspp & code
that used it.
Now for project file loading -- APSCpp.c:LoadProjectImages()
-- it already loads y,p,r, now change needed
-- make it parse PTAsm, PTGui style file names...
These are on comment lines of the form
#-imgfile 3522 2348 "_MG_6981.CR2"
rather than on 'i' or 'o' lines. Name may be full path.
20 July
debugging load pts and ptp projects..
ptp load ok, but pts has a "dummy image" o-line that makes
trouble. I think it can be handled just by skipping o-lines
that don't have a file name -- must reset FNflag at eof in
case 2 passes.
But it can't, because file o-lines have back references to
dummy image attributes. So we have to support a true dummy
image in the DIinfo list. Grrr. Let's do that by leaving the
file name null and inserting checks for that where needed.
That seems to work, however a,b,c from the dummy image line
are not making it to the output file (not unsurprising as we
don't read them, but we should).
Verrrry interesting.
Here's what three PT family stitchers report for the hfov
of two fisheye images, using EXIF data
stitcher FL crop rectilinear fisheye
Canon 15mm
PTAsm 5.0 15 1.6 73.74 85.94
Hugin 0.8 15 1.6 73.69 85.86
PTGui 8.1 15 1.587 74.19 91.67
Sigma 8mm
PTAsm 5.0 8 1.6 109.16 161.14
Hugin 0.8 8 1.6 109.1 161
PTGui 8.1 8 1.587 86.77 180
PTGui recognizes both lenses as fisheyes, PTAsm and Hugin
have to be told. But it gets the crop factor wrong (Hugin
has 1.60149, the value I compute from the 30D EXIF data).
PTGui uses the equal-area formula for fisheyes, the others
use the equal-angle. The big discrepancy on the 8mm fisheye
is because PTGui reports crop circle diameter for "circular"
fisheyes, all other fovs are for the short image dimension.
Fixed nasty bug in WritePTOFile(): using same index in outer
& inner loops. This could cause some i-lines to disappear.
To get a,b,c into the output:
--add them to DIinfo and project parser
--write them to output pto
Now deal with some differences between stitchers
1) PTAsm and PTGui give image dimensions in #-imgfile line,
so have to read those.
2) for circular fisheye, PTGui's hfov is the diameter of the
cropping circle. Hugin & PTAsm give fov for the horizontal
axis, even if that exceed crop circle. This ccould be fixed
at project read time by rescaling PTGui's number to hdim, if
I knew the crop circle diameter in pixels. But that info is
not apparent in the project file.
3) If the EXIF specifies an orientation, PTGui reports image
as oriented that way, regardless of filed orientation. Hugin
and PTAsm use the filed dimensions. This means we have to
check the filed dimentsions against the nominal ones when we
read the images, and adjust hfov if they don't agree.
4) PTAsm lets you specify fisheye type, using format 29 for
equal-area. Hugin treats all fisheyes as equal angle, PTGui
treats all as equal-area. APSCpp should let you choose.
21 July -- deal with stitcher differences...
-- read PTAsm and PTGui image dims like the file name;
-- adjust hfov at file read if DIinfo dims are transposed
(fatal error if they don't match either way).
-- provisionally assume PTGui's crop circle diameter equals
the long image axis.
TODO: correct this when better informed (have written Joost).
To deal with both fisheye types...
-- command option --equal-angle-fisheye sets default fisheye type
-- make CamLens treat formats 2,3 as equal-angle, 8 as equal-area;
-- make readProject identify Hugin, PTasm & PTGui projects and
accept PTAsm format 29 as equal-area fisheye (convert to 8),
and if not PTAsm, convert formats 2 & 3 to the default, and
adjust hfov if necessary.
-- print name of source format when converting to stg. (code is
already shown, but who knows what that means?)
Testing...
1) set up Hugin, PTAsm and PTGui projects for 3 circular fisheye
images, and same rotated 90 degrees. Build release APSCpp and
run against all 6 projects (equal area, no ransac) giving 6 pto's.
Load those in Hugin and compare alignments.
2) run same projects with default equal-area fisheye; compare
3) set up Hugin, PTAsm and PTGui projects for a rectilinear
image set, and its rotated form; run & compare alignments.
Found 2 problems:
1) writing out format 8, should have been put back to 2 or 3.
(how to tell which??)
2) FL reported by Hugin depends on originating stitcher, and
not quite right even when that was Hugin.
So--
-- at read project, compute fl in pixels & put that in DIinfo
then compute hfov from it where needed (incl. @ output)
-- review all the monkey motion with hfov, delete code where
possible.
And maybe someday....
-- add copy-project option
This is a goal left over from original revision, that seems
more desirable now: an option to output a copy of the whole
project file, with old CPs commented out and new ones added.
To comment-out use "#-old-CP "
Give the output file name the same extension as the input
project, no matter what user enters?
22 July Design for layout-guided CP selection
The key thing is to match keypoints for one pair of images
at a time. A list of pairs to be processed will be created,
based on overlaps predicted by the layout.
The overlaps should be evaluated using only (Y,P) and some
kind of average image diameter. Then candidate CPs can be
validated (for each pair) as follows.
1) map KP coordinates to the sphere (using lens fn, including
correction if known).
2) repeat {
find rotation of one image around optic axis that minimizes
the variance of KP differences (considered as 3-vectors);
discard outlier CPs;
} until( set is pretty compact );
3) If there are more than maxmatches CPs left, sort them on
position across the line between image centers, divide into
maxmatches groups, and keep the strongest CP in each group.
If there is no layout, it is more efficient to match KPs for
all images at once, as now; however for really big jobs it
might be necessary to divide the kp's into several groups &
do global matching on those. Anyhow, we should then select
the image pairs having decent numbers of matches and do the
rotational refinement on those.
It would be possible to construct a complete alignment from
the results of the pairwise alignments. But perhaps just a
consistency check is enough.
It might be feasible to use a Monte Carlo approach to
build the overlapping pairs list: Match a random selection
of keypoints (equal numbers from each image), count matches
by pair, use the pairs with above average numbers of matches.
This could be a good check for layout errors, too.
So I need a KP matching subroutine that will support all
these modes. For full generality it should take a list of
KP lists to be loaded into the k-d tree, and a second list
of KP lists to be used as probes, and output a list of CPs
(caller can separate that into lists by image pair).
And a rotational optimizer that takes a list of CPs for a
single pair of images; the projection specifications; and
maybe a tuning parameter or two, and returns the subset of
good CPs and the relative alignment of the pair. I don't
think this needs to be given an initial alignment.
Infrastructure---
I'd like to abandon Nowozin's MultiMatch structure as it
puts all KP data and CP data "in one basket", making it
hard to partition a big job pairwise. That will mean
abandoning his polishing fns too, as they all assume a
MultiMatch. So with this change the "autopano" part of
APSCpp will become a completely different program from
autopano.
For a performance boost on multicore systems, it would be
good to repackage the KP finder as a task (or a separate
command pgm).
But first, test whether there are any systematic coordinate
errors in CPs:
-- make project with an image and same rotated 90 degrees;
check optimization errors.
1) Hugin assigns different focal lengths to the 2 images, so
must set them equal by hand. It then stitches them with near
perfect cancellation, either with the 90 deg rotation set at
load, or using CPS from legacy APSCpp.
PFS_093-PFS_093-rot.pto, "8mm crop 1.60149 circular fisheye".
2)Run this thru APSCpp 2.5.2 giving
93-93r-stg.pto -- maxdim 1600, stg on
Hugin and PTAsm display CP at same spots in both images;
Hugin gets good auto alignment after deleting 2 bad CPS.
PTAsm auto alignment fails.
93-93r-nostg.pto -- maxdim 1600, stg off
Hug & Pta show CPs not coinciding,
Hug auto alignment very bad.
Clearly I am botching CP rescaling when the stg is off.
-- try re-using the pre-find remapper & (when stg off) inverting
the pre-find scale. Now both stg on & stg off CPs are misplaced!
The relative positions are same in both images, but all are in
the wrong places.
So must review how to prescale & scale back.... tomorrow.
23 July kp coord scaling tests w/rotated image pairs
-- put stg rescaling back the way it was. Compute true factor
from long image axis instead of width.
Now stg on gives good match w/90 deg rot (1 wild CP), stg off
still gives shifted CP sets.
-- try 180 deg rotation: PFS_093-PFS_093-180.pto
stg on: perfect alignment (1 bad point).
stg off: perfect alignment (2 bad points).
So it looks like N. code gets coords wrong on 90 deg rot, and
SAremap somehow corrects that?? No, as usual the fault is all
mine: I forgot to make undoing the stereographic mapping
conditional on having done it in the first place. With that
fixed the no-stg 90 deg case aligns perfectly (0 bad pnts) --
indeed significantly better than with stg enabled, which means
the remapping makes SIFT localization a bit less certain.
Now that it's working, improve the API some more....
-- lens-type option to override PT format codes.
-- focal-length + crop factor option to override hfovs.
-- lens-correction option to use a,b,c,d,e from project.
26 July BUT FIRST -- some more testing
Test set: L:EOSPix\StFx\IMG_7346-7350.pts. Six 15mm fisheye
pix that stitch well with PTGui. Run APSCpp with ransac on,
maxdim 1600 (2.19x reduction), autoalign with Hugin...
result CPs Avg Worst
PTGui autoalign 200 1.0 7.8
no stg 245 2.9 35.4
no stg + refine 249 3.5 307
stg 222 2.1 17.5
stg + refine 241 2.7 26.8
and at full source resolution...
no stg, full rez 334 2.9 20.6
stg, full rez 296 2.1 19.2
Stereographic remapping seems quite worthwhile, and gives
a big payoff per CPU second because it runs very fast.
Reducing resolution did not degrade CP quality measurably.
The refine option takes a very long time and makes the result
worse. It is actually an attempt to find SIFT CPs over again
at original resolution, in small patches around the found CPs.
I think I'll drop it, possibly in favor of something like local
correlation, perhaps on patches that have be reprojected to
on-axis rectilinear...
Anyhow the bottom line is that APSCpp's CPs are still far
worse than PTGui's. And it takes about 5 times longer to
find them.
BTW ransac does prune some bad points, but not enough, it
needs attention too.
27 July Let's get on with pairwise matching...
Set up an image pairs list first, use layout if available, else
list all pairs. Sort the list on 1st member to minimize the no.
of k-d tree builds. Match thru k-d tree giving a CP list for
each pair.
Do a 'rotational alignment' on each pair to cull out wild CPs.
This uses the kp coords mapped to sphere. Rotate one image around
Z to minimize the variance of the difference vectors across each CP
-- 3d variance is best because all good CPs will be parallel after
alignment. But the rotation is 2d of course. Use L-M unless an
analytic solution presents itself (surely one exists). Throw out
obvious outlier CPs and repeat, until there are no obvious outliers.
Using the pair alignment just computed, one could then refine kp
positions by remapping a patch around each CP from original images
to panospace and doing correlation (shifts only). Woth a try.
Finally, review the ransac procedure to see if it could be made to
use pairwise and/or global alignments effectively.