glean-dev — Glean development discussions

Re: [glean-dev] Only test one visual when --quick parameter is used

[Nicolai H. <pre...@gm...>]

On Monday 21 September 2009 17:47:37 Brian Paul wrote:
> OK, here's a patch series for review.
>
> When --quick but not --visuals is specified, testing is limited to the
> first rgb,z,stencil visual.

This is indeed a much better solution, thank you!

cu,
Nicolai

> -Brian


-- 
Lerne, wie die Welt wirklich ist,
aber vergiss niemals, wie sie sein sollte.

Re: [glean-dev] Only test one visual when --quick parameter is used

[Brian P. <br...@vm...>]

Allen Akin wrote:
> On Mon, Sep 21, 2009 at 09:47:37AM -0600, Brian Paul wrote:
>>> I guess I'm in the habbit of running glean on just one visual with the  
>>> --visual option when I want a quick run.
> 
> Same here.
> 
>> OK, here's a patch series for review.
>>
>> When --quick but not --visuals is specified, testing is limited to the  
>> first rgb,z,stencil visual.
> 
> Looks OK to me.  The only change I might suggest is making sure the
> visual is conformant as well as rgb,z,stencil.

Good idea.  Done.

-Brian

Re: [glean-dev] Only test one visual when --quick parameter is used

[Allen A. <ak...@ar...>]

On Mon, Sep 21, 2009 at 09:47:37AM -0600, Brian Paul wrote:
>> I guess I'm in the habbit of running glean on just one visual with the  
>> --visual option when I want a quick run.

Same here.

> OK, here's a patch series for review.
>
> When --quick but not --visuals is specified, testing is limited to the  
> first rgb,z,stencil visual.

Looks OK to me.  The only change I might suggest is making sure the
visual is conformant as well as rgb,z,stencil.

Allen

Re: [glean-dev] Only test one visual when --quick parameter is used

[Brian P. <br...@vm...>]

Brian Paul wrote:
> Nicolai Hähnle wrote:
>> A lot of drivers expose a large number of visuals. So even though the --quick 
>> option reduces the time taken by texCombine significantly, "quick" is still 
>> kind of the wrong adjective to describe glean with the --quick option.
> 
> I guess I'm in the habbit of running glean on just one visual with the 
> --visual option when I want a quick run.
> 
> 
>> The attached patch (which I've taken the liberty to apply to the copy of glean 
>> in piglit for now) changes the behaviour of --quick so that only the first 
>> selected visual gets selected, in the hope of making those test runs a bit 
>> less annoying for people.
>>
>> There's a potential pitfall here in that this might select a visual on which 
>> certain tests don't apply. On the limited systems I've tested here, the first 
>> visual is always double-buffered with Z buffer and stencil, so I hope this 
>> isn't a problem in practice.
> 
> I think we really need to make sure that a visual with at least a Z 
> buffer is chosen.  Otherwise, quite a few tests won't run.
> 
> I'll see if I can come up with a better patch.

OK, here's a patch series for review.

When --quick but not --visuals is specified, testing is limited to the 
first rgb,z,stencil visual.

-Brian

Re: [glean-dev] Only test one visual when --quick parameter is used

[Brian P. <br...@vm...>]

Nicolai Hähnle wrote:
> A lot of drivers expose a large number of visuals. So even though the --quick 
> option reduces the time taken by texCombine significantly, "quick" is still 
> kind of the wrong adjective to describe glean with the --quick option.

I guess I'm in the habbit of running glean on just one visual with the 
--visual option when I want a quick run.


> The attached patch (which I've taken the liberty to apply to the copy of glean 
> in piglit for now) changes the behaviour of --quick so that only the first 
> selected visual gets selected, in the hope of making those test runs a bit 
> less annoying for people.
> 
> There's a potential pitfall here in that this might select a visual on which 
> certain tests don't apply. On the limited systems I've tested here, the first 
> visual is always double-buffered with Z buffer and stencil, so I hope this 
> isn't a problem in practice.

I think we really need to make sure that a visual with at least a Z 
buffer is chosen.  Otherwise, quite a few tests won't run.

I'll see if I can come up with a better patch.

-Brian

[glean-dev] Only test one visual when --quick parameter is used

[Nicolai H. <pre...@gm...>]

A lot of drivers expose a large number of visuals. So even though the --quick 
option reduces the time taken by texCombine significantly, "quick" is still 
kind of the wrong adjective to describe glean with the --quick option.

The attached patch (which I've taken the liberty to apply to the copy of glean 
in piglit for now) changes the behaviour of --quick so that only the first 
selected visual gets selected, in the hope of making those test runs a bit 
less annoying for people.

There's a potential pitfall here in that this might select a visual on which 
certain tests don't apply. On the limited systems I've tested here, the first 
visual is always double-buffered with Z buffer and stencil, so I hope this 
isn't a problem in practice.

cu,
Nicolai
-- 
Lerne, wie die Welt wirklich ist,
aber vergiss niemals, wie sie sein sollte.

[glean-dev] new glean git URL

[Brian P. <br...@vm...>]

FYI: SourceForge has changed the URLs for the git repositories.

To switch to the new URL edit your glean/.git/config file:

Old:
url = ssh://USE...@gl.../gitroot/glean

New:
url = ssh://USE...@gl.../gitroot/glean/glean


-Brian

Re: [glean-dev] RFC: relax exactRGBA test tolerance

[Allen A. <ak...@ar...>]

The intent of the second paragraph of 2.14.9 is to call out a special
case that overrides the general behavior described in the other places
you mentioned.  Not a self-contradiction; just a special case that
applies only when all the conditions mentioned in the paragraph are met.

As the comments in runOne() indicate, other tests in glean may depend on
being able to write exact values into the framebuffer.  If you mask the
actual behavior by allowing more slop in ExactRGBA, you might see
failures in other tests that are more difficult to track down because
ExactRGBA appears to pass.  I don't have time to look at the moment, but
if any test explicitly examines the results from ExactRGBA, this problem
is likely to arise.

Allen

Re: [glean-dev] [Mesa3d-dev] RFC: relax exactRGBA test tolerance

[Brian P. <br...@vm...>]

Corbin Simpson wrote:
> On 07/15/2009 08:04 AM, Brian Paul wrote:
>> The attached patch relaxes the error tolerance for the exactRGBA test.
>>
>> This test fails with many (all?) GL implementations because the most
>> signficant M bits of values passed to glColor4us() and glColor4ui()
>> don't always match the M bits found in the framebuffer.  There's
>> typically a 1-bit max error.  For example, the GLushort value 0xba00
>> will wind up as the framebuffer value 0xb9.
>>
>> The GL spec is a bit self-contradictory here.  It says how GLushort and
>> GLuint colors are converted to floats and it says how float color
>> components are converted to framebuffer values (by rounding to nearest
>> int).  But if you follow those rules, it's not possible to achive the
>> "M-bit identity" rule.
>>
>> The attach patch allows a 1-bit error for the glColor4us() and
>> glColor4ui() functions.
>>
>> Comments?
> 
> Seems fine for now. We really need a language clarification, though.

You (or anyone else) can provide spec feedback at
http://www.opengl.org/documentation/specs/feedback/

I've got a little too much on my plate right now to persue this myself.

-Brian

[glean-dev] RFC: relax exactRGBA test tolerance

[Brian P. <br...@vm...>]

The attached patch relaxes the error tolerance for the exactRGBA test.

This test fails with many (all?) GL implementations because the most 
signficant M bits of values passed to glColor4us() and glColor4ui() 
don't always match the M bits found in the framebuffer.  There's 
typically a 1-bit max error.  For example, the GLushort value 0xba00 
will wind up as the framebuffer value 0xb9.

The GL spec is a bit self-contradictory here.  It says how GLushort 
and GLuint colors are converted to floats and it says how float color 
components are converted to framebuffer values (by rounding to nearest 
int).  But if you follow those rules, it's not possible to achive the 
"M-bit identity" rule.

The attach patch allows a 1-bit error for the glColor4us() and 
glColor4ui() functions.

Comments?

-Brian

Re: [glean-dev] [glean-patches] glean (master): Use newlines and PASS/FAIL for multi test results

[Allen A. <ak...@ar...>]

On Thu, Jun 18, 2009 at 11:14:24AM +0100, José Fonseca wrote:
| Actually they were impossible for a computer to read too, since there
| was no white space between numbers (at least on windows). :)

It's definitely a good idea to make the change, then. :-)

| IMO, something also worth thinking is using a more structured result
| file (it could be a standard XML/YAML format, but it could be just our
| own format) which is able both of carrying both objective
| machine-readable data (e.g., name, visual info, passes/failures,
| precision, performance measures, etc.), but also subjective
| human-readable (e.g., any additional information, warning messages,
| etc ). This allows all this info to be stored in a single place, so that
| other tools can aggregate and present in a website in an automated
| fashion.

Agreed.

Allen

Re: [glean-dev] [glean-patches] glean (master): Use newlines and PASS/FAIL for multi test results

[José F. <jfo...@vm...>]

On Wed, 2009-06-17 at 17:08 -0700, Allen Akin wrote:
> On Wed, Jun 17, 2009 at 10:53:57AM +0000, Jose Fonseca wrote:
> | Module: glean
> | Branch: master
> | Commit: 91525da8f920c2f9b67b6556069de55870ee80e3
> | URL:    no.cgit.repo/commit/?id=91525da8f920c2f9b67b6556069de55870ee80e3
> | 
> | Author: José Fonseca <jfo...@vm...>
> | Date:   Wed Jun 17 10:40:04 2009 +0100
> | 
> | Use newlines and PASS/FAIL for multi test results
> | 
> | Otherwise results just pile up, and are impossible to read.
> 
> I don't object to this change, but I do want to mention that my original
> intent was that humans would never read the contents of a results file.
> In theory, the test log should contain everything that's needed for a
> human to understand testing, comparing different runs, and debugging, in
> a text format that's either easily human-readable or (e.g. HTML) could
> be converted automatically to something easily human-readable.

Actually they were impossible for a computer to read too, since there
was no white space between numbers (at least on windows). :)

> Glean falls short of this goal in many ways, so it is sometimes useful
> to be able to read a results file.  

The reason why I look at results sometimes is that the stdout/stderr is
often cluttered with the mesa warnings, debugging output, etc, which is
also good to have when you're diagnosing a failure, but when you're just
interested at checking if a change improves/regresses anything,
comparing results is the best.

> But the solution I had in mind for
> the long run would be to require that every result of a test go into the
> results file, change the logging code so that it didn't depend on
> information that's transient or not recorded in the results file, and
> factor out the logging code so that it could be used with a new
> command-line option to produce human-readable logs from the results
> files.

I see. Yes, I also think that's a good goal.

IMO, something also worth thinking is using a more structured result
file (it could be a standard XML/YAML format, but it could be just our
own format) which is able both of carrying both objective
machine-readable data (e.g., name, visual info, passes/failures,
precision, performance measures, etc.), but also subjective
human-readable (e.g., any additional information, warning messages,
etc ). This allows all this info to be stored in a single place, so that
other tools can aggregate and present in a website in an automated
fashion.

Jose

Re: [glean-dev] [glean-patches] glean (master): Use newlines and PASS/FAIL for multi test results

[Allen A. <ak...@ar...>]

On Wed, Jun 17, 2009 at 10:53:57AM +0000, Jose Fonseca wrote:
| Module: glean
| Branch: master
| Commit: 91525da8f920c2f9b67b6556069de55870ee80e3
| URL:    no.cgit.repo/commit/?id=91525da8f920c2f9b67b6556069de55870ee80e3
| 
| Author: José Fonseca <jfo...@vm...>
| Date:   Wed Jun 17 10:40:04 2009 +0100
| 
| Use newlines and PASS/FAIL for multi test results
| 
| Otherwise results just pile up, and are impossible to read.

I don't object to this change, but I do want to mention that my original
intent was that humans would never read the contents of a results file.
In theory, the test log should contain everything that's needed for a
human to understand testing, comparing different runs, and debugging, in
a text format that's either easily human-readable or (e.g. HTML) could
be converted automatically to something easily human-readable.

Glean falls short of this goal in many ways, so it is sometimes useful
to be able to read a results file.  But the solution I had in mind for
the long run would be to require that every result of a test go into the
results file, change the logging code so that it didn't depend on
information that's transient or not recorded in the results file, and
factor out the logging code so that it could be used with a new
command-line option to produce human-readable logs from the results
files.

Just FYI.

Allen

[glean-dev] new --quick option

[Brian P. <br...@vm...>]

I've added a --quick option to Glean.  It causes some tests (only 
texCombine for now) to run fewer iterations in order to reduce the 
execution time.

If anybody wants additional tests to observe the --quick flag let me 
know...

-Brian

Re: [glean-dev] Moving from CVS to Git

[Allen A. <ak...@ar...>]

On Wed, Apr 01, 2009 at 10:01:18AM +0800, Shuang He wrote:
| Hi, Allen
|     Could you please tell us where's the git tree?

Hi, Shuang!

My apologies.  The repository is still on SourceForge, so their
conventions apply.  The easiest way to get a read-only tree is:

	git clone git://glean.git.sourceforge.net/gitroot/glean

and a read-write tree (for which you should use the same SourceForge
username you previously used for CVS access):

	git clone ssh://USE...@gl.../gitroot/glean

Allen

Re: [glean-dev] Moving from CVS to Git

[Shuang He <shu...@in...>]

Hi, Allen
    Could you please tell us where's the git tree?

Thanks
    --Shuang

Allen Akin wrote:
> Hi, everyone.
>
> We've begun the process of moving glean from CVS control to Git.  The
> repository has been converted, Git access enabled, and CVS access
> disabled (to keep things from getting out of sync in the short term).
>
> Development traffic on glean is very low, and I expect all glean
> developers are familiar with git, so I don't anticipate any problems,
> but if you need access to the CVS repository re-enabled, just let me
> know.  We'll figure out how to sync the repositories as needed.
>
> I'll be updating the documentation in the next day or two.
>
> Thanks to Jose Fonseca for doing the heavy lifting.
>
> Allen
>
> ------------------------------------------------------------------------------
> _______________________________________________
> glean-dev mailing list
> gle...@li...
> https://lists.sourceforge.net/lists/listinfo/glean-dev
>   

[glean-dev] Moving from CVS to Git

[Allen A. <ak...@po...>]

Hi, everyone.

We've begun the process of moving glean from CVS control to Git.  The
repository has been converted, Git access enabled, and CVS access
disabled (to keep things from getting out of sync in the short term).

Development traffic on glean is very low, and I expect all glean
developers are familiar with git, so I don't anticipate any problems,
but if you need access to the CVS repository re-enabled, just let me
know.  We'll figure out how to sync the repositories as needed.

I'll be updating the documentation in the next day or two.

Thanks to Jose Fonseca for doing the heavy lifting.

Allen

Re: [glean-dev] [glean-announce] a test for texture parameters/filters

[Brian P. <br...@vm...>]

Nir Radian wrote:
>  
> 
> Hi!
> 
> I was wondering if there was such a test that goes over all texture
> filter types, testure wrap types, lod bias, and so forth…I couldn’t find
> one – seems to me this could be very useful.

I don't think we have tests for those specific things.

Texture coord clamping and filtering are areas where we sometimes find differences from one GL to another.  For example, differences in sub-pixel positioning, LOD calculation and texture border clamping might not be noticable in typical apps but can cause grief for testing.  You may need somewhat "loose" tolerances for some tests.

If you're interested in adding other tests, here's a few I'd really like to see:

1. Test all possible texture sizes (in a variety of formats) to be sure the layout of textures in GPU memory is correct.

2. Test max-size textures and FBOs.  Create the largest possible texture maps and largest possible framebuffer objects and make sure sampling/rendering is correct.

If you're interested in any of those, there's some Mesa test programs that could be used as starting points.

-Brian

------------------------------------------------------------------------------
_______________________________________________
glean-announce mailing list
gle...@li...
https://lists.sourceforge.net/lists/listinfo/glean-announce

[glean-dev] [glean-announce] a test for texture parameters/filters

[Nir R. <ni...@ho...>]

------------------------------------------------------------------------------

Re: [glean-dev] Which Version of OpenGL does Glean support?

[Claire Li <Yu...@Su...>]

Allen Akin wrote:
> On Fri, Nov 28, 2008 at 10:59:38AM +0800, Claire Li wrote:
> |        Could anybody tell me which version of OpenGL spec does the latest
> |    release(1.1) of Glean support? I didn't find the exact answer in the CVS
> |    code....
>
> Glean has never attempted to have full API coverage the way the
> conformance tests are supposed to, so it isn't tied to OpenGL versions
> in the same way as the conformance tests.  Instead, as Alan said,
> development is driven mostly be the need to test individual feature
> correctness or regressions or performance more thoroughly than the
> conformance tests can.
>
> Usually, the biggest part of the work is developing individual tests
> rather than the infrastructure.  There are occasional exceptions (for
> example, exposing framebuffer objects as drawing surfaces might be one).
>
> |        And is there a schedule for the release time of 1.2?
>
> I don't have one.  For a major project with a lot of users, a formal
> release schedule and support plan is necessary.  For a minor project
> whose users are mostly developers, like glean, it didn't appear to be
> worth the cost in developer time.  So I recommend just using the latest
> CVS version.
>
> Allen
>   
I see. Thanks for your clear answer!

Best Regards,
Claire

Re: [glean-dev] [glean-announce] Which Version of OpenGL does Glean support?

[Claire Li <Yu...@Su...>]

Alan Hourihane wrote:
> On Fri, 2008-11-28 at 10:59 +0800, Claire Li wrote:
>   
>> Hi all,
>>
>>     Could anybody tell me which version of OpenGL spec does the latest
>> release(1.1) of Glean support? I didn't find the exact answer in the
>> CVS code....
>>     And is there a schedule for the release time of 1.2?
>>     
>
> I'm not sure when a release will happen, but glean is advancing. And
> rather try not to think of glean supporting an OpenGL version, but
> rather testing specific pieces.
>
> There are lots of extensions in OpenGL which glean can test quite a few,
> and you can find that out with the --listtests option.
>   

Sounds cool. I will try the --listtests option.

Thanks very much!
Claire

> Alan.
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> glean-announce mailing list
> gle...@li...
> https://lists.sourceforge.net/lists/listinfo/glean-announce
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> glean-dev mailing list
> gle...@li...
> https://lists.sourceforge.net/lists/listinfo/glean-dev
>   

Re: [glean-dev] Which Version of OpenGL does Glean support?

[Allen A. <ak...@ar...>]

On Fri, Nov 28, 2008 at 10:59:38AM +0800, Claire Li wrote:
|        Could anybody tell me which version of OpenGL spec does the latest
|    release(1.1) of Glean support? I didn't find the exact answer in the CVS
|    code....

Glean has never attempted to have full API coverage the way the
conformance tests are supposed to, so it isn't tied to OpenGL versions
in the same way as the conformance tests.  Instead, as Alan said,
development is driven mostly be the need to test individual feature
correctness or regressions or performance more thoroughly than the
conformance tests can.

Usually, the biggest part of the work is developing individual tests
rather than the infrastructure.  There are occasional exceptions (for
example, exposing framebuffer objects as drawing surfaces might be one).

|        And is there a schedule for the release time of 1.2?

I don't have one.  For a major project with a lot of users, a formal
release schedule and support plan is necessary.  For a minor project
whose users are mostly developers, like glean, it didn't appear to be
worth the cost in developer time.  So I recommend just using the latest
CVS version.

Allen

Re: [glean-dev] [glean-announce] Which Version of OpenGL does Glean support?

[Alan H. <al...@fa...>]

On Fri, 2008-11-28 at 10:59 +0800, Claire Li wrote:
> Hi all,
> 
>     Could anybody tell me which version of OpenGL spec does the latest
> release(1.1) of Glean support? I didn't find the exact answer in the
> CVS code....
>     And is there a schedule for the release time of 1.2?

I'm not sure when a release will happen, but glean is advancing. And
rather try not to think of glean supporting an OpenGL version, but
rather testing specific pieces.

There are lots of extensions in OpenGL which glean can test quite a few,
and you can find that out with the --listtests option.

Alan.

[glean-dev] Which Version of OpenGL does Glean support?

[Claire Li <Yu...@Su...>]

Hi all,

    Could anybody tell me which version of OpenGL spec does the latest 
release(1.1) of Glean support? I didn't find the exact answer in the CVS 
code....
    And is there a schedule for the release time of 1.2?

Thanks and Best Regards!
Claire

Re: [glean-dev] new case for FBO

[Brian P. <bri...@tu...>]

He, Shuang wrote:
> He, Shuang wrote:
>> Hi,
>>        I've written a glean case for GL_EXT_framebuffer_object. The 
>> test case is attached.
>>
>> The test case is categorized into following parts:
>>
>>                  Functionality:
>>
>>                     Sanity check (checks max number of supported color 
>> attachment points, default framebuffer binding, max Renderbuffer Size)
>>
>>                     Render to texture through single Framebuffer 
>> object (1D, 2D, 3D, Cube texture, combined with depth/stencil buffer).
>>
>>                     Render to multiple textures through 
>> multiple/single Framebuffer object (test render to 2D texture in 
>> different usage models).
>>
>>                     Render to depth texture without color buffer.
>>
>>                     Render to texture with manual mipmap generation.
>>
>>            Performance:
>>
>>                     Compare performance of new style texture through 
>> FBO with previous glCopyTexImgage style texture
>>
>> Thanks
>>    --Shuang
>>
>>
>>
> Updated test case attached. since I forgot to set the texture wrap 
> modes. And test case can get pass/fail=8/0 for software rendering now.

Thanks!  I've committed it to the cvs repo.

-Brian