pyopengl-users

[PyOpenGL-Users] More difficulties with framebuffers

[Derakon <de...@gm...>]

After my previous success in getting framebuffers to run at all in my
program, I spent some time getting them to work *properly*...and I'm
stumped. To recap:

I have a program that uses a camera to take images of the contents of
a microscope slide. These images are tiled in a mosaic viewer, and the
user can pan and zoom about in the viewer. Once we get a few thousand
tiles, the viewer starts bogging down because it has to try to draw
all of those tiles when the user is zoomed out. I want to modify the
viewer to pre-render the tiles using framebuffers at a low level of
detail (packing many tiles onto each pre-rendered megatile); then,
when the user zooms out, I can switch from rendering each tile
individually to using the pre-rendered megatiles with no apparent loss
of detail but a reduction in orders of magnitude of the number of
textures that OpenGL has to deal with.

Unfortunately, I'm running into some scaling and offsetting issues --
the megatiles render too small; I need to scale them up by a factor of
about 1.958 to get them close to the right size. Obviously they should
be rendering in exactly the right spot, but since I don't know why
their scale is off I don't know what the proper fix is (the 1.958 is
just a hack to get them approximately right).

I've uploaded a standalone app that shows off the problem here:

http://derakon.dyndns.org/~chriswei/temp/mosaicapp.tgz

It depends on PyOpenGL, wx, and numpy. I'm using the framebuffers in
GL.EXT here (old version of OpenGL).

Controls:
Left-click: prints out location of click in canvas space.
Alt-left-click: pan
Shift-alt-left-click: zoom
Right-click: spawn a new tile

The megatiles include some debugging rendering; every 400 units of
canvas space, they render some text marking the location, as well as a
single pixel marking the exact location. If you zoom in on some text
you should see the point to the left of the left parenthesis; you can
click on that to compare what the megatile thinks the location is to
what the canvas knows the location is.

As far as code is concerned, mosaicViewer.GLViewer.OnPaint(),
mosaicTile.MegaTile.prerenderTiles(), and mosaicTile.MegaTile.render()
should have everything; the rest is just infrastructure. I've uploaded
just those three functions to a pastebin here:

http://paste.ubuntu.com/544652/

This is a mishmash of very old code and code I've written myself;
sorry about the style clash. I'm doing my best to clean things up as I
come to understand what they are.

If any of you have any ideas about what could be going wrong, I'd love
to hear them.

-Chris

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Derakon <de...@gm...>]

Er, my mistake: the megatiles render too *big* and I have to scale
them down. Each one renders in the right location (the lower-left
corner of the megatile is the same location in "canvas space" and
"megatile space") but is too large, so other pixels don't map
properly.

-Chris

On Thu, Dec 16, 2010 at 3:15 PM, Derakon <de...@gm...> wrote:
> After my previous success in getting framebuffers to run at all in my
> program, I spent some time getting them to work *properly*...and I'm
> stumped. To recap:
>
> I have a program that uses a camera to take images of the contents of
> a microscope slide. These images are tiled in a mosaic viewer, and the
> user can pan and zoom about in the viewer. Once we get a few thousand
> tiles, the viewer starts bogging down because it has to try to draw
> all of those tiles when the user is zoomed out. I want to modify the
> viewer to pre-render the tiles using framebuffers at a low level of
> detail (packing many tiles onto each pre-rendered megatile); then,
> when the user zooms out, I can switch from rendering each tile
> individually to using the pre-rendered megatiles with no apparent loss
> of detail but a reduction in orders of magnitude of the number of
> textures that OpenGL has to deal with.
>
> Unfortunately, I'm running into some scaling and offsetting issues --
> the megatiles render too small; I need to scale them up by a factor of
> about 1.958 to get them close to the right size. Obviously they should
> be rendering in exactly the right spot, but since I don't know why
> their scale is off I don't know what the proper fix is (the 1.958 is
> just a hack to get them approximately right).
>
> I've uploaded a standalone app that shows off the problem here:
>
> http://derakon.dyndns.org/~chriswei/temp/mosaicapp.tgz
>
> It depends on PyOpenGL, wx, and numpy. I'm using the framebuffers in
> GL.EXT here (old version of OpenGL).
>
> Controls:
> Left-click: prints out location of click in canvas space.
> Alt-left-click: pan
> Shift-alt-left-click: zoom
> Right-click: spawn a new tile
>
> The megatiles include some debugging rendering; every 400 units of
> canvas space, they render some text marking the location, as well as a
> single pixel marking the exact location. If you zoom in on some text
> you should see the point to the left of the left parenthesis; you can
> click on that to compare what the megatile thinks the location is to
> what the canvas knows the location is.
>
> As far as code is concerned, mosaicViewer.GLViewer.OnPaint(),
> mosaicTile.MegaTile.prerenderTiles(), and mosaicTile.MegaTile.render()
> should have everything; the rest is just infrastructure. I've uploaded
> just those three functions to a pastebin here:
>
> http://paste.ubuntu.com/544652/
>
> This is a mishmash of very old code and code I've written myself;
> sorry about the style clash. I'm doing my best to clean things up as I
> come to understand what they are.
>
> If any of you have any ideas about what could be going wrong, I'd love
> to hear them.
>
> -Chris
>

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Christopher B. <Chr...@no...>]

On 12/16/10 3:15 PM, Derakon wrote:
> a microscope slide. These images are tiled in a mosaic viewer, and the
> user can pan and zoom about in the viewer. Once we get a few thousand
> tiles, the viewer starts bogging down because it has to try to draw
> all of those tiles when the user is zoomed out. I want to modify the
> viewer to pre-render the tiles using framebuffers at a low level of
> detail (packing many tiles onto each pre-rendered megatile); then,
> when the user zooms out, I can switch from rendering each tile
> individually to using the pre-rendered megatiles with no apparent loss
> of detail but a reduction in orders of magnitude of the number of
> textures that OpenGL has to deal with.
>
> Unfortunately, I'm running into some scaling and offsetting issues --

Sorry, no time to try to take a look at your code, but we have a similar 
system in maproom, for rending tiles at different zoom levels, maybe it 
will help you figure out your issue:

https://bitbucket.org/dhelfman/maproom/wiki/Home

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Christopher B. <Chr...@no...>]

On 12/16/10 4:26 PM, Christopher Barker wrote:
> Sorry, no time to try to take a look at your code, but we have a similar
> system in maproom, for rending tiles at different zoom levels, maybe it
> will help you figure out your issue:
>
> https://bitbucket.org/dhelfman/maproom/wiki/Home

I think this is the relevant file:

https://bitbucket.org/dhelfman/maproom/src/668ea464624e/maproomlib/plugin/Tile_set_layer.py

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Derakon <de...@gm...>]

I took a look at your code, but it looks like you're doing this all
"manually" while I'm trying to leverage OpenGL to do the work for me.
The basic flow of my code is:

Main render function:
  if zoomed far out:
    render each megatile in view
  else:
    render each individual tile in view

Add new tile function:
  find megatile(s) that this tile overlaps
    render tile to megatiles

Something is going wrong in the "render to megatile" code, since when
I render the megatile later in the main render function, its scale is
off. But I can't figure out why.

-Chris

On Thu, Dec 16, 2010 at 4:31 PM, Christopher Barker
<Chr...@no...> wrote:
> On 12/16/10 4:26 PM, Christopher Barker wrote:
>>
>> Sorry, no time to try to take a look at your code, but we have a similar
>> system in maproom, for rending tiles at different zoom levels, maybe it
>> will help you figure out your issue:
>>
>> https://bitbucket.org/dhelfman/maproom/wiki/Home
>
> I think this is the relevant file:
>
> https://bitbucket.org/dhelfman/maproom/src/668ea464624e/maproomlib/plugin/Tile_set_layer.py
>
> -Chris
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chr...@no...
>

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Christopher B. <Chr...@no...>]

Sorry for the slow response -- lots going on here.

Derakon wrote:
> I took a look at your code, but it looks like you're doing this all
> "manually" while I'm trying to leverage OpenGL to do the work for me.

I'm not sure that's true. We do break the process down into two stages:

1) generate the tiles at different zoom levels.

2) render them.

1) is done without using OpenGL at all. Perhaps we should have used it, 
  but we've had in mind being able to use different rendering back-ends, 
for instance for PDF generation or printing, so we wanted to b able to 
have the tiles without any OpenGL.

We've also separated the generation of tiles from rendering because we 
may well want to use tiles that are pre-generated, such as from 
OpenStreetMap and the like.

2) The rendering stage is using OpenGL as much as I know how. For a 
given zoom level, we need to determine which scale of tiles to use, 
which specific tiles to use, how they are scaled to OpenGL coords, then 
pass them in. Isn't that what you are doing?


> The basic flow of my code is:
> 
> Main render function:
>   if zoomed far out:
>     render each megatile in view
>   else:
>     render each individual tile in view

That's pretty much what we do, except we have many levels of zoom -- it 
looks like you have only two.

> Add new tile function:
>   find megatile(s) that this tile overlaps
>     render tile to megatiles

ahh -- that is a bit different. rather than renderintg each tile each 
time, you render to a megatile, then just render that. What is the 
advantage of that, why not just render the tiles directly? I suppose 
that does take a Python loop, but I don't think that's very slow in this 
case. Our tiles are 256X256, so for a 1000X1000 screen, that's 16 tiles 
--not many at all.

> Something is going wrong in the "render to megatile" code, since when
> I render the megatile later in the main render function, its scale is
> off. But I can't figure out why.

It is a real pain to get the math right for that kind of thing. All I 
can suggest is some really tedious debugging/print statements, and check 
the math at each  step.

good luck,

-Chris



> -Chris
> 
> On Thu, Dec 16, 2010 at 4:31 PM, Christopher Barker
> <Chr...@no...> wrote:
>> On 12/16/10 4:26 PM, Christopher Barker wrote:
>>> Sorry, no time to try to take a look at your code, but we have a similar
>>> system in maproom, for rending tiles at different zoom levels, maybe it
>>> will help you figure out your issue:
>>>
>>> https://bitbucket.org/dhelfman/maproom/wiki/Home
>> I think this is the relevant file:
>>
>> https://bitbucket.org/dhelfman/maproom/src/668ea464624e/maproomlib/plugin/Tile_set_layer.py


-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Derakon <de...@gm...>]

I did eventually get the zoom factor right. Turns out it's determined
by the ratio of the size of the tile in pixels (512) and the size of
the tile in OpenGL units (1000). Which makes sense; it just required a
bunch of fiddling to sort out. Thanks for the help all.

More stuff inline.

On Wed, Dec 22, 2010 at 10:29 AM, Christopher Barker
<Chr...@no...> wrote:
> Sorry for the slow response -- lots going on here.
>
> Derakon wrote:
>> The basic flow of my code is:
>>
>> Main render function:
>>  if zoomed far out:
>>    render each megatile in view
>>  else:
>>    render each individual tile in view
>
> That's pretty much what we do, except we have many levels of zoom -- it
> looks like you have only two.

Yeah, our use cases are basically 1) looking at details of tiles we've
imaged earlier, and 2) zooming way out so we can pan around quickly to
zoom in on other tiles. So there's no real need for intermediate zoom
levels. I've set up the megatiles to be able to hold about 150 normal
tiles (so a ~10x decrease in resolution) and it looks just fine.

>
>> Add new tile function:
>>  find megatile(s) that this tile overlaps
>>    render tile to megatiles
>
> ahh -- that is a bit different. rather than renderintg each tile each time,
> you render to a megatile, then just render that. What is the advantage of
> that, why not just render the tiles directly? I suppose that does take a
> Python loop, but I don't think that's very slow in this case. Our tiles are
> 256X256, so for a 1000X1000 screen, that's 16 tiles --not many at all.

I'm not certain I understand your question. The function I'm
describing here is the "we've just taken a new image with our camera;
now we need to add it to the mosaic" function. That also implies that
we need to add it to any appropriate megatiles, so that later when we
draw the megatile, the new image will be there.

With the old behavior , we were rendering each tile that intersected
the view every frame. This meant that when you zoomed far out, we had
to iterate over thousands of tiles and render them, which turned out
to be costly. The goal is to avoid having to tell OpenGL to render
large numbers of textures, which is accomplished by pre-rendering to
the megatiles. Then each frame, we draw the megatiles to the screen,
instead of drawing the normal tiles. When we're zoomed in close enough
to see details, then we still render the normal tiles directly, but
that's cheap because we can cull out all the tiles that don't
intersect the view.

>
>> Something is going wrong in the "render to megatile" code, since when
>> I render the megatile later in the main render function, its scale is
>> off. But I can't figure out why.
>
> It is a real pain to get the math right for that kind of thing. All I can
> suggest is some really tedious debugging/print statements, and check the
> math at each  step.
>

Boy howdy, you ain't kidding. At least it works now.

> -Chris
>

-Chris

Re: [PyOpenGL-Users] More difficulties with framebuffers

[Christopher B. <Chr...@no...>]

On 12/22/10 5:25 PM, Derakon wrote:
> I did eventually get the zoom factor right. Turns out it's determined
> by the ratio of the size of the tile in pixels (512) and the size of
> the tile in OpenGL units (1000). Which makes sense; it just required a
> bunch of fiddling to sort out.

yup -- seems simple in retrospect, but hard to get right.


> Yeah, our use cases are basically 1) looking at details of tiles we've
> imaged earlier, and 2) zooming way out so we can pan around quickly to
> zoom in on other tiles. So there's no real need for intermediate zoom
> levels.

well, it's a matter of how big a range of zoom you need to support.

> I've set up the megatiles to be able to hold about 150 normal
> tiles (so a ~10x decrease in resolution) and it looks just fine.

I'm sure it will look fine, with GL's texture scaling built in, perform 
fine.

In our case, we are doing a set of tiles for each factor-of-two zoom 
level. That's probably a lot more than necessary, however. WE did that 
because that' a standard way to tile map images. IN the case we have 
currently implemented, they re just re-scaled versions of the same 
image, so no good reason to do it. However, it is common with maps to 
have the maps rendered with different levels of detail at different zoom 
levels, so you really do need all those layers.

See google maps for an example.

It sounds like you have a fine solution for your problem.

> I'm not certain I understand your question. The function I'm
> describing here is the "we've just taken a new image with our camera;
> now we need to add it to the mosaic" function. That also implies that
> we need to add it to any appropriate megatiles, so that later when we
> draw the megatile, the new image will be there.

I think it's similar -- except that you have full resolution tiles, and 
  a 1/10 resolution tile, where as we have a bigger set -- a set of 
tiles at full resolution, one at 1/2 resolution, 1/4, etc, all the way 
down to whatever gives us a single 256x256 tile.

> With the old behavior , we were rendering each tile that intersected
> the view every frame. This meant that when you zoomed far out, we had
> to iterate over thousands of tiles and render them, which turned out
> to be costly. The goal is to avoid having to tell OpenGL to render
> large numbers of textures,

yup.

> Boy howdy, you ain't kidding. At least it works now.

Good work -- it's satisfying when it does work right!

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...