We are developing a molecular modeling application which basically boils
down to drawing of the order 10000-60000 spheres when viewing the larger
system. So far we render spheres by adding a unit gluSphere to a display
list and then scaling and positioning this appropriately for each atom
in the system.
I am aware that display lists are old-school. So my questions boils down
to this: What is the defacto fastest way of rendering a sphere in a
modern OpenGL framework?
There's nothing wrong with display lists. It's just that they're technically deprecated in future version of OpenGL. However, because so much code uses display lists, and because it's overall a very good way of doing things, I predict support for them won't drop for at least another decade. Display lists can also have beer performance than vertex buffer objects (VBOs), which are the current standard.
VBOs are almost as fast as display lists, but also support a number of fun features, especially dynamically changing just *sections* of the data.
However, what you're doing is suboptimal to begin with. 10,000-60,000 spheres is a lot of geometry. Even if you only had 12 vertices per sphere (icosahedron), that's still around 120,000-720,000 vertices, which is . . . doable, but only just. Transforming each sphere to the right position will take FAR too long individually (10,000-60,000 glTranslate and 10,000-60,000 glScale calls each draw will really eat your framerate).
As I see it, you have two general solutions:
A) Load a single display list (or VBO) containing ALL the spheres positioned and scaled correctly.
B) Store the desired positions and scalars of each atom as single pixels in a texture. For each sphere, give each of its vertice a vertex attribute corresponding to the sphere's position/scale in the texture. Draw your 10,000-60,000 spheres in the same place (probably with a DL or VBO), and then a shader will take care of moving them to the correct place and scaling them appropriately, based on the information stored in the texture.
The second option has the advantage that, by changing the texture, you can move the atoms around--you might even write a shader to do this, and have a full molecular simulation running on the GPU!
Finally, for an extra speed kick, you can use impostors (i.e., point sprites with an image of a sphere on them). This will cut the number of vertices per sphere to 1, which will VASTLY improve performance. If you write a shader to output depth, and discard fragments outside the sphere, intersection with other objects will work exactly the way it would if the atoms where actual spheres. Note that doing this procedure is applicable to both A and B.