Thread: [PyOpenGL-Users] Pyopengl - slow performance using
Brought to you by:
mcfletch
From: <pm...@gm...> - 2011-01-13 11:29:36
|
Hi all, I am developing an application using pyopenGL to read data from a shape file and to draw it in 2D using openGl. However, the performance is really bad; it takes up to 20 seconds to draw the map. I am using Ubuntu and nVidia NVS 3100M. The shape file being read has the following characteristics: - 50 000 entries containing the description roads segments; each road segment has the associated geometry (set of (x,y) points ) - +/- 350 000 points in total A simplified version of the code is shown bellow: a) to get data from shapefile (using pyshapelib) and store it in a list of lists of tuples (python): ... self.shp = [] self.extents = [] self.array = [] ... self.shp = shapelib.ShapeFile(filename) for i in range(self.shp.info()[0]): lineVert = self.shp.read_object(i).vertices() tempArray = [] for j in lineVert[0]: tempArray.append([j[0],j[1]]) self.array.append(tempArray) b) to draw the data using pyopenGL: ... glEnableClientState(GL_VERTEX_ARRAY) # obtained from the shapefile(array element) arrayList = self.map.getArrayList() if len(arrayList) > 0: for objMap in arrayList: self.drawPolyline(objMap) def drawPolyline(self,inArray): #glColor(1.0, 0.0, 0.0) glVertexPointerf(inArray) glDrawArrays(GL_LINE_STRIP, 0, len(inArray)) - Do you have any hints how to improve the performance dramatically and make it usable? I tried several options but none worked out. For instance I tried to replace python list with numpy arrays but the performance was not significantly changed. - Do I need execute the OpenGL code in C? Altough I could do this, I still need to use python as scripting language due to other dependencies of the implementation. Thank you in advance for your help! Best Regards, Pedro -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de |
From: Alejandro S. <as...@gm...> - 2011-01-13 11:52:35
|
Hello Pedro, On Jan 13, 2011, at 9:29 AM, pm...@gm... wrote: > Hi all, > > I am developing an application using pyopenGL to read data from a shape file and to draw it in 2D using openGl. However, the performance is really bad; it takes up to 20 seconds to draw the map. I am using Ubuntu and nVidia NVS 3100M. > > The shape file being read has the following characteristics: > > - 50 000 entries containing the description roads segments; each road segment has the associated geometry (set of (x,y) points ) > - +/- 350 000 points in total > > > A simplified version of the code is shown bellow: > > a) to get data from shapefile (using pyshapelib) and store it in a list of lists of tuples (python): > ... > self.shp = [] > self.extents = [] > self.array = [] > ... > self.shp = shapelib.ShapeFile(filename) > for i in range(self.shp.info()[0]): > lineVert = self.shp.read_object(i).vertices() > tempArray = [] > for j in lineVert[0]: > tempArray.append([j[0],j[1]]) > self.array.append(tempArray) > > b) to draw the data using pyopenGL: > ... > glEnableClientState(GL_VERTEX_ARRAY) > # obtained from the shapefile(array element) > arrayList = self.map.getArrayList() > if len(arrayList) > 0: > for objMap in arrayList: > self.drawPolyline(objMap) > > def drawPolyline(self,inArray): > #glColor(1.0, 0.0, 0.0) > glVertexPointerf(inArray) > glDrawArrays(GL_LINE_STRIP, 0, len(inArray)) > > - Do you have any hints how to improve the performance dramatically and make it usable? I tried several options but none worked out. For instance I tried to replace python list with numpy arrays but the performance was not significantly changed. I find it puzzling that you didn't notice any speedup at all after doing this. Python lists are just linked lists, which means they are probably not consecutive in memory, but rather, they have to be converted to an array at each call to drawArrays. Are you 100% certain you were converting from a python list to a numpy array just once and not at every frame? The other thing I noticed is that you are making several calls to drawArrays. This might be required by your app's logic, but if data is actually static, I would consider making just one big array and supplying all the data at once to OpenGL. Finally, I would also consider using VBOs to prevent copying vertex data from main memory to the GPU several times over (specially if your data is static). > > - Do I need execute the OpenGL code in C? Altough I could do this, I still need to use python as scripting language due to other dependencies of the implementation. I don't think so, as long as you avoid doing a lot of calculations using python and you using deprecated features, you should be fine. Please consider revising the usage of numpy arrays. Good luck! Alejandro.- > > Thank you in advance for your help! > > Best Regards, > > Pedro > > -- > Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir > belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de > > ------------------------------------------------------------------------------ > Protect Your Site and Customers from Malware Attacks > Learn about various malware tactics and how to avoid them. Understand > malware threats, the impact they can have on your business, and how you > can protect your company and customers by using code signing. > http://p.sf.net/sfu/oracle-sfdevnl > _______________________________________________ > PyOpenGL Homepage > http://pyopengl.sourceforge.net > _______________________________________________ > PyOpenGL-Users mailing list > PyO...@li... > https://lists.sourceforge.net/lists/listinfo/pyopengl-users |
From: Jonathan H. <ta...@ta...> - 2011-01-13 12:16:30
|
On 13/01/2011 11:50, Alejandro Segovia wrote: > I find it puzzling that you didn't notice any speedup at all after doing this. Python lists are just linked lists, which means they are probably not consecutive in memory, but rather, they have to be converted to an array at each call to drawArrays. Hey. I'm guessing this implies there is something else going on which is massively slower, swamping the change from lists to numpy arrays. This hypothesis is borne out by the reported performance of 20 secs per frame. :-) -- Jonathan Hartley Made of meat. http://tartley.com ta...@ta... +44 7737 062 225 twitter/skype: tartley |
From: Jonathan H. <ta...@ta...> - 2011-01-13 12:16:30
|
On 13/01/2011 11:29, pm...@gm... wrote: > Hi all, > > I am developing an application using pyopenGL to read data from a shape file and to draw it in 2D using openGl. However, the performance is really bad; it takes up to 20 seconds to draw the map. I am using Ubuntu and nVidia NVS 3100M. > > The shape file being read has the following characteristics: > > - 50 000 entries containing the description roads segments; each road segment has the associated geometry (set of (x,y) points ) > - +/- 350 000 points in total > > > A simplified version of the code is shown bellow: > > a) to get data from shapefile (using pyshapelib) and store it in a list of lists of tuples (python): > ... > self.shp = [] > self.extents = [] > self.array = [] > ... > self.shp = shapelib.ShapeFile(filename) > for i in range(self.shp.info()[0]): > lineVert = self.shp.read_object(i).vertices() > tempArray = [] > for j in lineVert[0]: > tempArray.append([j[0],j[1]]) > self.array.append(tempArray) > > b) to draw the data using pyopenGL: > ... > glEnableClientState(GL_VERTEX_ARRAY) > # obtained from the shapefile(array element) > arrayList = self.map.getArrayList() > if len(arrayList)> 0: > for objMap in arrayList: > self.drawPolyline(objMap) > > def drawPolyline(self,inArray): > #glColor(1.0, 0.0, 0.0) > glVertexPointerf(inArray) > glDrawArrays(GL_LINE_STRIP, 0, len(inArray)) > > - Do you have any hints how to improve the performance dramatically and make it usable? I tried several options but none worked out. For instance I tried to replace python list with numpy arrays but the performance was not significantly changed. > > - Do I need execute the OpenGL code in C? Altough I could do this, I still need to use python as scripting language due to other dependencies of the implementation. > > Thank you in advance for your help! > > Best Regards, > > Pedro > Hey there, Have you profiled or timed your code to see precisely which parts are taking all the time? Personally I highly recommend using RunSnakeRun to draw diagrams from the output of stdlib cProfile. Let me know if you'd like to do this but are having problems, I'll help you out. Since GIS data is mostly static (i.e. not animated), so that you don't need to process every vertex every frame, I would be very surprised if you had to move from Python to C for performance reasons. I am confident that you will be able to optimise this code to be fast. I'd guess that part (a) of the code, constructing the arrays, is probably pretty expensive, since you are processing every individual vertex. Importantly, this code should only need to be run at application start-up, not every frame, is this correct? Also, in part (b) of the code, where you loop over the objects in arrayList: for objMap in arrayList: self.drawPolyline(objMap) I can see that the value of objMap represents a vertex array, but how many vertices are generally in it? You may find substantial acceleration from increasing the number of vertices in an array, and hence decreasing the number of times this loop needs to iterate. In order to do this, you will need to put the arrays for several roads into a single array. I don't know whether it will be useful to put the vertices for *all* roads into a single array - there may be an upper limit on the useful size of vertex arrays. Doubtless someone cleverer than me on the list will know this. Also, a very minor point: if len(arrayList)> 0: for objMap in arrayList: self.drawPolyline(objMap) If arrayList is a normal Python iterable (e.g. a list), then this could be written more simply as: for objMap in arrayList: self.drawPolyline(objMap) Best regards, Jonathan -- Jonathan Hartley Made of meat. http://tartley.com ta...@ta... +44 7737 062 225 twitter/skype: tartley |
From: Chris B. <Chr...@no...> - 2011-01-13 22:49:13
|
On 1/13/2011 3:29 AM, pm...@gm... wrote: > I am developing an application using pyopenGL to read data from a shape file and to draw it in 2D using openGl. However, the performance is really bad; it takes up to 20 seconds to draw the map. I am using Ubuntu and nVidia NVS 3100M. You may be interested in this: https://bitbucket.org/dhelfman/maproom/wiki/Home It aims to be a general purpose map viewer/manipulator library for custom mapping applications, written in Python, with PyOpenGL, and some Cython for performance bottlenecks. We don't have shapefile line segment support yet, but the pieces are all there, so it wouldn't be hard to add, and we've addressed your performance issues. How far are you going with this? Maybe an opportunity for collaboration? On 1/13/2011 3:50 AM, Alejandro Segovia wrote: > I find it puzzling that you didn't notice any speedup at all after > doing this. Python lists are just linked lists, no, they are not -- they are internally regular C arrays. > which means they are > probably not consecutive in memory, but rather, they have to be > converted to an array at each call to drawArrays. however, they are arrays of pointers to PyObjects, so you are right, each pyObject needs to be converted, and that can be slow, you don't want to do that with each pain event. On 1/13/2011 3:56 AM, Jonathan Hartley wrote: > Personally I highly recommend using RunSnakeRun to > draw diagrams from the output of stdlib cProfile. cool! thanks, I hadn't see that before -- looks really useful. > I'd guess that part (a) of the code, constructing the arrays, is > probably pretty expensive, since you are processing every individual > vertex. Importantly, this code should only need to be run at application > start-up, not every frame, is this correct? or at data loading. MapRoom is a bit pokey when you load a large data set, but then the drawing is plenty fast. > I can see that the value of objMap represents a vertex array, but how > many vertices are generally in it? You may find substantial acceleration > from increasing the number of vertices in an array, Yup -- I think this is your proplem -- you won't want to have to draw each line segment separately -- I'll bet you have a lot. As a rule, if you have much looping in Python in side your drawing code, it's going to be slow. > the number of times this loop needs to iterate. In order to do this, you > will need to put the arrays for several roads into a single array. yup -- I *think* you can put NaNs in between segments to create a gap and do a bunch with one line drawing call. or, if you know which segments are connected, you ca re-join them, essentially. HTH, -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Ian M. <geo...@gm...> - 2011-01-13 23:17:18
|
Hi, If you're binding a new vertex array for each individual line, then that is certainly your problem. I once had an application where I changed vertex buffers a couple times per object. Maybe 50 bindings, and the program was too slow to be usable. The problem had to be solved with interleaved vertex arrays/VBOs. 350,000 vertex buffer bindings, and I see why you have performance issues! You can solve this by batching more of your edges in a single vertex array. If you batched all of them into a single vertex array, I'd predict somewhere around 5-15fps. Because you're still drawing these things with vertex arrays--i.e., transferring all the data across the graphics bus each frame. Even if you have to duplicate vertices, you'll still be far better off. AFTER you've done this, to make your application realtime, you can then transition to VBOs, where you'll transfer the data once. I recommend using the OpenGL.arrays.vbo class. VBOs are the "best" solution, and will even let you do fancy stuff like update the data (or only a part of it). An easier solution is using display lists--although they're technically depreciated in versions of OpenGL that no nobody likes. After this, you'll probably be around 50-70fps. Keep in mind that 350,000 * lines* is a hefty load, because each line (internally) requires two triangles. On Thu, Jan 13, 2011 at 3:49 PM, Chris Barker <Chr...@no...> wrote: > On 1/13/2011 3:50 AM, Alejandro Segovia wrote: > > I find it puzzling that you didn't notice any speedup at all after > > doing this. Python lists are just linked lists, > > no, they are not -- they are internally regular C arrays. > But they are resizable, which implies they are either linked lists or array lists--because I doubt the pointers are copied over to a brand-new array just big enough each time a single element is added or removed. Ian |
From: Chris B. <Chr...@no...> - 2011-01-14 02:45:26
|
On 1/13/2011 3:17 PM, Ian Mallett wrote: > > doing this. Python lists are just linked lists, > > no, they are not -- they are internally regular C arrays. > > But they are resizable, which implies they are either linked lists or > array lists--because I doubt the pointers are copied over to a brand-new > array just big enough each time a single element is added or removed. A bit OT, but they handle re-sizing by over allocating when appended to. So most of the time you can append to a list without any memory allocation or copying, but as it grows, it does need to do that once in a while. linked lists would be great for appending, but horrible for indexing and slicing. But anyway, the original point is correct -- python lists don't store the data in a way that is compatible with OpenGL, so a lot of work has to be done to create OpenGL objects from lists -- numpy arrays are really just wrappers around blocks of memory, so they can efficiently be used to move data in and out of OpenGL. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Ian M. <geo...@gm...> - 2011-01-14 03:46:51
|
On Thu, Jan 13, 2011 at 7:45 PM, Chris Barker <Chr...@no...> wrote: > On 1/13/2011 3:17 PM, Ian Mallett wrote: > > > doing this. Python lists are just linked lists, > > > > no, they are not -- they are internally regular C arrays. > > > > But they are resizable, which implies they are either linked lists or > > array lists--because I doubt the pointers are copied over to a brand-new > > array just big enough each time a single element is added or removed. > > A bit OT, but they handle re-sizing by over allocating when appended to. > So most of the time you can append to a list without any memory > allocation or copying, but as it grows, it does need to do that once in > a while. > Right--this data structure is called an array list. You're absolutely correct that Python users should avoid it for very high performance. I usually stick with NumPy arrays for all but the most dead simple stuff. They also tend to be compatible with other packages--in particular, I've used them to tie very nicely into PyOpenGL's VBO class: my_vbo = vbo.VBO(numpy_1x3_array,'f') In fact, they seemed to be the only thing that actually did work. And, speed aside, NumPy provides some truly great functionality when it comes to working with arrays--like element-wise operations, anyone? Ian |
From: Chris B. <Chr...@no...> - 2011-01-15 01:23:53
|
On 1/14/2011 12:22 PM, Pedro Miranda wrote: > But is someone sure that the proposed strategy to use > NaN to separate different road segments is viable? Actually, no, sorry -- now that I think about it more, I think we found that NaNs behaved differently on different Video Cards, so that may not be safe. > [[(-45005.21, 168113.25), (-45003.77, 168112.71), (-44995.30, > 168109.057)]] > > How can I convert this lineVert to a numpy array and create a "big" > array containing all lineVerts? numpy arrays are fixed size, so your best bet is to create a big list first, and the convert to numpy arrays. Something like: for i in xrange(num_segments): lineVert.extend(self.shp.read_object(i).vertices()) line_array = np.array(lineVert, dtype=np.float32) That will put them all sequentially together. In real use, you will want to either: create one numpy array for each connected set of segments -- you can do that by seeing if the endpoint of one is the startpoint of the next or Keep track of the indexes where the breaks occur, and I think you can then put it all in a VBO or VertexArray, and index into that to draw each contiguous line. Sorry, I didn't write the OpenGL part of MapRoom, so I have only a hazy idea of how this works. > Is there any upper limit to maximum number of vertices a vertex array > can contain? probably only memory limits. > d) VBO ------ The next step in the development of the application > (after the map layer is smoothly drawn in openGL) is to draw a series > of routes on top of the map (static). However, in this case this data > can be more dynamic because the user can select the time interval of > the data that is shown. I will definitely consider in that case using > VBO. Though I suspect that the route data is MUCH less, so performance may not be an issue -- VBOS may be a good way to do anyway. \> e) number of vertices per objMap (vertex array) > ----------------------------------------------- I was checking and > there are : - 7 vertices in average per vertex array - 2 vertices is > the minimum number of vertices per vertex array - 272 vertices is the > maximum mumber of vertices per vertex array > > So the average is pretty low and making so many calls to openGL > routines is certainly good. yes -- that is you key problem, you need to join the ones together that are contiguous, at least. I'd test that and see what the numbers look like. > f) maproom ---------- > > Thank you for sending the link of maproom. I will install it and > check how it works. The API contains lots of useful functionality! > Collaboration seems a good idea; let's see how thinks evolve. OK -- it's been sleeping a bit after Dan Helfman left us -- but we are picking it up again, and probably hiring a new developer to work on it -- anyone looking for work in Seattle? I'll try to see if I can add loading of shape files like yours to it next week -- we do want that eventually. > FYI: I have recently found that Quantum GIS has python bindings as > well. You can find more information in > http://www.qgis.org/pyqgis-cookbook/ Yup -- QGIS is really nice, and can be used as a toolkit for building custom apps -- we're not using it because we have data that does not fit into the standard GIS data model, and we want to be able to draw fast and interactrively edit big data sets -- QGIS doesn't (or didn't when we evaluated it) do that very well. Perhaps it's a good option for you, though. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Christopher B. <Chr...@no...> - 2011-01-18 20:27:28
|
On 1/17/11 2:24 PM, Jonathan Hartley wrote: > Personally I use ctypes arrays instead of numpy arrays. > But for what it's worth, the ctypes code is quite > simple. From memory, assuming 3D vertices: > > from OpenGL import GL as gl > > # prepare the arrays once at application start-up or model load: > > verts = [ (1.0, 2.0, 3.0), (4.0, 5.0, 6.0)... ] > length = len(verts) * 3 # each vertex has three ordinates, (x, y, z) > arraytype = gl.GLfloat * length > glverts = arraytype(*verts) sure but compare that to: verts = [ (1.0, 2.0, 3.0), (4.0, 5.0, 6.0)... ] glverts = np.array(verts, dtype=np.float32) but if you're not using numpy for anything else, ctypes arrays are a fine option. > If anyone anyone else's response contradicts the above, they are almost > certainly more correct. :-) Maybe, but that was really helpful info and pointers, thanks! -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Pedro M. <pm...@gm...> - 2011-01-14 20:23:08
|
Hi all, First of all thank you for your all your comments and fast replies. Second, I am sorry for late replying but I was away for the day. I should also add that I am not professional programmer and I was not aware of how some mechanisms work in programming. In order to avoid spamming the mailing list I will try to address all the comments you have sent in a single e-mail. 1) Numpy arrays --------------- Previously to starting the tread I tried to make an implementation using numpy arrays but I guess I was not successful. As many of you point out this should be the way to go for managing GIS data. In our comments you state that I could I could make a "big" numpy array containing all the road segments. For me this is ok because this is just map data that I will not be changing it (static). But is someone sure that the proposed strategy to use NaN to separate different road segments is viable? The following code to read a line of the shapefile returns a list of vertices for a given road segment: lineVert = self.shp.read_object(i).vertices() A print of lineVert will give for example: [[(-45005.21, 168113.25), (-45003.77, 168112.71), (-44995.30, 168109.057)]] How can I convert this lineVert to a numpy array and create a "big" array containing all lineVerts? Is there any upper limit to maximum number of vertices a vertex array can contain? b) Python List creation ----------------------- I am just loading the shapefile and creating the list of lists of tuples at the initialization of the application. Since this is static data I just do this once at the beginning of the application. I do not allow the user to load a new map during runtime; besides this is not interesting for my application. c) profiling ------------ I was not aware of the tool RunSnakeRun to do profiling in python. Thank you for sending this! It was useful! Analysing the output I could find out that the majority of the time (almost 11 seconds or 60% of the time) was spent in function "as...@li...", which is a part of python/OpenGL/Arrays. I think this function does the conversion that have been mentioned here. The second most important cost was due to "wra...@wr...". If someone is interested I can send a copy of the profile file. d) VBO ------ The next step in the development of the application (after the map layer is smoothly drawn in openGL) is to draw a series of routes on top of the map (static). However, in this case this data can be more dynamic because the user can select the time interval of the data that is shown. I will definitely consider in that case using VBO. Do someone know any good tutorial to use VBO in dynamic scenarios? e) number of vertices per objMap (vertex array) ----------------------------------------------- I was checking and there are : - 7 vertices in average per vertex array - 2 vertices is the minimum number of vertices per vertex array - 272 vertices is the maximum mumber of vertices per vertex array So the average is pretty low and making so many calls to openGL routines is certainly good. f) maproom ---------- Thank you for sending the link of maproom. I will install it and check how it works. The API contains lots of useful functionality! Collaboration seems a good idea; let's see how thinks evolve. FYI: I have recently found that Quantum GIS has python bindings as well. You can find more information in http://www.qgis.org/pyqgis-cookbook/ Thank you! Pedro -------- Original-Nachricht -------- > Datum: Thu, 13 Jan 2011 20:46:45 -0700 > Von: Ian Mallett <geo...@gm...> > An: Chris Barker <Chr...@no...> > CC: pyopengl-users <pyo...@li...> > Betreff: Re: [PyOpenGL-Users] Pyopengl - slow performance using > On Thu, Jan 13, 2011 at 7:45 PM, Chris Barker <Chr...@no...> > wrote: > > > On 1/13/2011 3:17 PM, Ian Mallett wrote: > > > > doing this. Python lists are just linked lists, > > > > > > no, they are not -- they are internally regular C arrays. > > > > > > But they are resizable, which implies they are either linked lists or > > > array lists--because I doubt the pointers are copied over to a > brand-new > > > array just big enough each time a single element is added or removed. > > > > A bit OT, but they handle re-sizing by over allocating when appended to. > > So most of the time you can append to a list without any memory > > allocation or copying, but as it grows, it does need to do that once in > > a while. > > > Right--this data structure is called an array list. > > You're absolutely correct that Python users should avoid it for very high > performance. I usually stick with NumPy arrays for all but the most dead > simple stuff. They also tend to be compatible with other packages--in > particular, I've used them to tie very nicely into PyOpenGL's VBO class: > > my_vbo = vbo.VBO(numpy_1x3_array,'f') > > In fact, they seemed to be the only thing that actually did work. And, > speed aside, NumPy provides some truly great functionality when it comes > to > working with arrays--like element-wise operations, anyone? > > Ian -- Empfehlen Sie GMX DSL Ihren Freunden und Bekannten und wir belohnen Sie mit bis zu 50,- Euro! https://freundschaftswerbung.gmx.de |
From: Jonathan H. <ta...@ta...> - 2011-01-17 22:24:53
|
I sent this a few days ago, but don't see it in my news feed of the group. Apologies if anyone else has seen it twice. On 14/01/2011 20:22, Pedro Miranda wrote: > 1) Numpy arrays > --------------- Personally I use ctypes arrays instead of numpy arrays. Numpy may be a better option, because then you have the option to use the fancy numpy operators on the data. But for what it's worth, the ctypes code is quite simple. From memory, assuming 3D vertices: from OpenGL import GL as gl # prepare the arrays once at application start-up or model load: verts = [ (1.0, 2.0, 3.0), (4.0, 5.0, 6.0)... ] length = len(verts) * 3 # each vertex has three ordinates, (x, y, z) arraytype = gl.GLfloat * length glverts = arraytype(*verts) # then later in your render loop: gl.glVertexPointer( 3, # each vertex has three ordinates gl.GL_FLOAT, 0, glverts ) > ... But is someone sure that the proposed strategy to use NaN to > separate different road segments is viable? I have never personally used NaN in my vertex array, so can't speak directly about that, but I have done very similar things using 'primitive restarts', which seem to be the same idea but for indexed vertex arrays. Indexed vertex arrays may be a good idea in any case, once you have fixed whatever your current performance problem is. If you fancy primitive restarts, see glEnable(GL_PRIMITIVE_RESTART) and glPrimitiveRestartIndex(), plus the detailed description in the OpenGL SuperBible, under 'Advanced Geometry Management / Drawing a lot of Geometry Efficiently / Combining Geometry Using Primitive Restart' (p494 of the fifth edition.) Alternatively, and possibly simpler, you may be able to achieve the same thing as primitive restart or NaN vertices, by using glMultiDrawArrays or similar. Again, this would require converting to use indexed arrays, but it will allow you to draw many primitives using a single call. SuperBible says that for some drivers this sacrifices a smidge of performance, since internally it is literally converted into a series of glDrawArrays calls, but I'd guess that should be a small problem compared to the performance issues of calling from Python in the first place. > Is there any upper limit to maximum number of vertices a vertex array > can contain? Experimenting with my own code right now (which is rubbish on almost all axes) with a single vertex array full of GL_TRIANGLES, I get: triangles fps 168,000 60 1,440,000 5 3,840,000 memory error This is on a modest pre-2005 era laptop, with color and normal arrays as well as vertex arrays. The memory error is in my own code at application start-up, creating intermediate lists before creating the vertex arrays - presumably the vertex arrays could be larger if my start-up code wasn't so lame. Regarding the performance of the above, my scene isn't culled in any way, and my triangles are fairly large, so I have tremendous overdraw in my scene, and am therefore fill-rate limited - presumably this will be less of an issue for you since roads are thin and 2D data doesn't have so much overdraw. > c) profiling > ------------ > the majority of the time (almost 11 seconds or 60% of the time) was > spent in function "as...@li..." You don't explicitly say whether this fixes your performance problem, or just gives you some good info. Are you still at 20 seconds per frame? In particular, I don't think you should need to be calling asArray at all after models are loaded. Have I misunderstood? > Do someone know any good tutorial to use VBO in dynamic scenarios? One tutorial idea is the shaders tutorial in the PyOpenGL documentation. This starts out by explaining and setting up VBOs: http://pyopengl.sourceforge.net/context/tutorials/index.xhtml However, it uses the PyOpenGL VBO class, which handles all the low-level work for you. If you want to know what's going on under the covers, see the source of class VBO class, somewhere in the middle of: http://bazaar.launchpad.net/~mcfletch/pyopengl/trunk/view/head:/OpenGL/arrays/vbo.py So then you could continue using PyOpenGL.VBO class in your own code, or you could write your own equivalent if you didn't want the dependency on all of PyOpenGL. If anyone anyone else's response contradicts the above, they are almost certainly more correct. :-) Jonathan -- Jonathan Hartley Made of meat. http://tartley.com ta...@ta... +44 7737 062 225 twitter/skype: tartley |
From: Mike C. F. <mcf...@vr...> - 2011-01-18 05:10:23
|
On 11-01-13 06:29 AM, pm...@gm... wrote: ... > - Do you have any hints how to improve the performance dramatically and make it usable? I tried several options but none worked out. For instance I tried to replace python list with numpy arrays but the performance was not significantly changed. The most effective speedup I can think of would be to pre-render the static geometry into (mostly transparent) image-maps and then using a single textured quad to present it. Modern cards handle large image maps pretty well, and a single textured quad should render pretty fast. If you're letting users do a lot of zooming you'll want your zoom-UI to be regenerating the texture according to some heuristic regarding how different the current zoom is from that used to generate the texture. On "release" of the zoom widget you'd regenerate again for the final zoom. > - Do I need execute the OpenGL code in C? Altough I could do this, I still need to use python as scripting language due to other dependencies of the implementation. Even with 350,000 vertices, using VBOs for the indices and the vertices should produce an almost-C-speed render. That is, define a vertex array, an index array that says which vertices to render, and then use a single call to render the whole set. If your card can pull that much geometry, I'm guessing that should be faster than 50,000 calls to render. HTH, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |
From: Christopher B. <Chr...@no...> - 2011-01-18 20:31:20
|
On 1/17/11 8:50 PM, Mike C. Fletcher wrote: > The most effective speedup I can think of would be to pre-render the > static geometry into (mostly transparent) image-maps and then using a > single textured quad to present it. Modern cards handle large image > maps pretty well, and a single textured quad should render pretty fast. Wow, I"m surprised -- it seems OpenGL should be well suited to this kind of thing (that is, simple vector drawing). But I suppose what you're talking about is basically double buffering, which is an age-old technique. However, if you've going to go that route, you might want to pre-render much prettier rasters with Mapnik or MapServer, and then use those. > If you're letting users do a lot of zooming you'll want your zoom-UI to > be regenerating the texture according to some heuristic regarding how > different the current zoom is from that used to generate the texture. > On "release" of the zoom widget you'd regenerate again for the final zoom. hmmm... that does let you get the zooming and double buffering, both. I"ll need to consider this for some of our stuff, though vector rending as been working well for us so far. > Even with 350,000 vertices, using VBOs for the indices and the vertices > should produce an almost-C-speed render. That is, define a vertex > array, an index array that says which vertices to render, and then use a > single call to render the whole set. If your card can pull that much > geometry, I'm guessing that should be faster than 50,000 calls to render. much, I think -- this approach is working well for us. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
From: Mike C. F. <mcf...@vr...> - 2011-01-24 07:16:05
|
On 11-01-18 03:31 PM, Christopher Barker wrote: > On 1/17/11 8:50 PM, Mike C. Fletcher wrote: >> The most effective speedup I can think of would be to pre-render the >> static geometry into (mostly transparent) image-maps and then using a >> single textured quad to present it. Modern cards handle large image >> maps pretty well, and a single textured quad should render pretty fast. > Wow, I"m surprised -- it seems OpenGL should be well suited to this kind > of thing (that is, simple vector drawing). But I suppose what you're > talking about is basically double buffering, which is an age-old technique. > > However, if you've going to go that route, you might want to pre-render > much prettier rasters with Mapnik or MapServer, and then use those. OpenGL is reasonably well suited to this kind of thing, but modern graphics cards focus more on textured throughput than pushing huge numbers of vertices. On my GeForce 7600 I get ~90fps for the kind of data-set we are talking about (from VBOs with a single call to glMultiDrawArrays). With regular (numpy) arrays that tops out at ~40fps. With the Mesa software renderer, 2fps. Workstation drivers will tend to be better at optimizing this kind of thing, but it's still a heck of a lot of geometry to push through the pipeline. A single textured quad only generates 6 vertices (though, of course, it generates a fragment per pixel)... >> If you're letting users do a lot of zooming you'll want your zoom-UI to >> be regenerating the texture according to some heuristic regarding how >> different the current zoom is from that used to generate the texture. >> On "release" of the zoom widget you'd regenerate again for the final zoom. > hmmm... that does let you get the zooming and double buffering, both. > I"ll need to consider this for some of our stuff, though vector rending > as been working well for us so far. Honestly I don't do this kind of stuff myself as my playing tends to be focused on "wandering around in 3D", where caching generally works better at the pre-computed geometry level. >> Even with 350,000 vertices, using VBOs for the indices and the vertices >> should produce an almost-C-speed render. That is, define a vertex >> array, an index array that says which vertices to render, and then use a >> single call to render the whole set. If your card can pull that much >> geometry, I'm guessing that should be faster than 50,000 calls to render. > much, I think -- this approach is working well for us. Which reminds me, we have a long-standing wart that glMultiDrawElements is implemented in Python at the moment... really should revisit that issue (pointer type auto-wrapping didn't work very well there). Have fun, Mike -- ________________________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://www.vrplumber.com http://blog.vrplumber.com |