scidvspc-users

[Scidvspc-users] CQL port to Scid vs PC

[Lionel <hay...@gm...>]

I've ported Lewis Stiller's and Gady Costeff's CQL v5.1 implementation 
(http://www.gadycosteff.com/cql) to Scid vs PC as a fully integrated 
search engine.

I and a couple of others have been beating up on the engine for 10-12 
days and it appears to be quite solid. The port was relatively trivial, 
so I'm not expecting many problems, but I'm looking for more volunteers 
to help test what is a rather extensive grammar, especially on non-Linux 
platforms.

In particular, it would be very helpful if someone possessing Harold van 
der Heijden's study database (http://hhdbv.nl/) could test the port 
against that db. There is a set of scripts in the examples directory to 
aid in that endeavor. I've done my best to find or generate games which 
test the edge cases which are not likely to be present in one's typical 
chess database, but Stiller recommends Heijden's db as something of a 
testing high bar.

The build process for including the CQL engine in the Scid executable 
has only been implemented for the Linux platform, as I have no way of 
testing on other platforms.  I will happily incorporate patches for the 
Makefiles for other platforms as they are offered.  The build requires a 
C++11 compiler. The CQL engine has been successfully compiled with GNU 
g++ and with clang, as well as with Microsoft Visual C++.  The entire 
Scid+CQL build has been tested with GNU g++.

I very strongly recommend that users of this feature read the CQL Search 
section in the Help Guide before getting started. There are some 
essential guidelines in that section which can save one from any number 
of major headaches and points of confusion.  Read all of it.  Twice.

Please contact me off list for the [rather large] CQL patch, which 
applies cleanly to the v4.18.1 tarball.  To avoid spamming this list 
with CQL-related noise, please address all questions/problems directly 
to me.  Further notices, clarifications, faqs, update patches, etc., 
coming from me will be addressed only to those who have requested the patch.

I'll be welcoming comments and feedback and suggestions while I'm 
eliminating the last of the minor memory leaks in the engine over the 
next few weeks, and hope to release a final patch toward the end of the 
year.  I personally find this integrated CQL engine an 
order-of-magnitude more satisfying to use than the standalone version, 
and the language, itself, continues to impress the hell out of me.


Many thanks go to Stiller and Costeff for their innovative work on CQL, 
and to Steve Atkinson for his long years of dedication to the Scid vs PC 
project.


-- Lionel

Re: [Scidvspc-users] CQL port to Scid vs PC

[Alan W. <a.c...@gm...>]

Hi Lionel,

Sorry for the naive questions, but to clarify:

1) You are providing the patch that needs to be applied to the 4.18 tarball.

2) The CQL implementation will use scidvspc's built-in search gui, so no 
cli lines needed?

3) If it proves to be a better search function, should Steve port it to 
the next version?

On 11/15/2017 07:10 PM, Lionel wrote:
>
> I've ported Lewis Stiller's and Gady Costeff's CQL v5.1 implementation 
> (http://www.gadycosteff.com/cql) to Scid vs PC as a fully integrated 
> search engine.
>
> I and a couple of others have been beating up on the engine for 10-12 
> days and it appears to be quite solid. The port was relatively 
> trivial, so I'm not expecting many problems, but I'm looking for more 
> volunteers to help test what is a rather extensive grammar, especially 
> on non-Linux platforms.
>
> In particular, it would be very helpful if someone possessing Harold 
> van der Heijden's study database (http://hhdbv.nl/) could test the 
> port against that db. There is a set of scripts in the examples 
> directory to aid in that endeavor. I've done my best to find or 
> generate games which test the edge cases which are not likely to be 
> present in one's typical chess database, but Stiller recommends 
> Heijden's db as something of a testing high bar.
>
> The build process for including the CQL engine in the Scid executable 
> has only been implemented for the Linux platform, as I have no way of 
> testing on other platforms.  I will happily incorporate patches for 
> the Makefiles for other platforms as they are offered.  The build 
> requires a C++11 compiler. The CQL engine has been successfully 
> compiled with GNU g++ and with clang, as well as with Microsoft Visual 
> C++.  The entire Scid+CQL build has been tested with GNU g++.
>
> I very strongly recommend that users of this feature read the CQL 
> Search section in the Help Guide before getting started. There are 
> some essential guidelines in that section which can save one from any 
> number of major headaches and points of confusion.  Read all of it.  
> Twice.
>
> Please contact me off list for the [rather large] CQL patch, which 
> applies cleanly to the v4.18.1 tarball.  To avoid spamming this list 
> with CQL-related noise, please address all questions/problems directly 
> to me.  Further notices, clarifications, faqs, update patches, etc., 
> coming from me will be addressed only to those who have requested the 
> patch.
>
> I'll be welcoming comments and feedback and suggestions while I'm 
> eliminating the last of the minor memory leaks in the engine over the 
> next few weeks, and hope to release a final patch toward the end of 
> the year.  I personally find this integrated CQL engine an 
> order-of-magnitude more satisfying to use than the standalone version, 
> and the language, itself, continues to impress the hell out of me.
>
>
> Many thanks go to Stiller and Costeff for their innovative work on 
> CQL, and to Steve Atkinson for his long years of dedication to the 
> Scid vs PC project.
>
>
> -- Lionel
>
> ------------------------------------------------------------------------------ 
>
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Scidvspc-users mailing list
> Sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scidvspc-users
>

Re: [Scidvspc-users] CQL port to Scid vs PC

[Lionel <hay...@gm...>]

On 11/15/2017 08:36 PM, Alan Whiteman wrote:
>
> Sorry for the naive questions, but to clarify:
>
> 1) You are providing the patch that needs to be applied to the 4.18
> tarball.

The 4.18.1 tarball, though either one will probably do just fine.

>
> 2) The CQL implementation will use scidvspc's built-in search gui, so no
> cli lines needed?
>

Correct.

> 3) If it proves to be a better search function, should Steve port it to
> the next version?
>

Depends on how you define better, and that's Steve's call.  There are 
good arguments to be made pro and con.  For any given query of 
equivalence, the native Scid query (header, material, pattern, etc), 
will be more efficient.  But the existing Scid search capability doesn't 
come close to the range and depth of the complexity allowed by a CQL query.


-- Lionel

Re: [Scidvspc-users] CQL port to Scid vs PC

[Steve A <ste...@gm...>]

Thanks to Lionel for this. We're working on it, and seems to be going ok :)

It's a very useful feature - only really limited by it's d*mn script
complexity - and will hopefully get a release in ScidvsPC 4.19 before *too*
many months. It should be in subversion in the next week or two.

I might just mention a few things. This is CQL 5.1. It is not really
compatible with CQL 3 , nor (sadly) with Scidb's new implementation.. so
keep this in mind when looking for example scripts. Here is my first/only
script :) , which finds: white pawn forks both black rooks.
    cql ()  shifthorizontal shiftvertical {ra8 rc8 Pb7}
Most of the existing doco is pretty ordinary, so we will be writing some
modest doco of our own.

Like Lionel mentions, it is possible to easily add *lots* of comments to a
database on accident
(eg "cql () r r r" does not find three black rooks, laugh)
but this will not be the *default* behaviour in ScidvsPC when we commit it
to svn. One handy feature this allows, is currently Scid is limited to a
single search match in any one game, but by (optionally) adding comments to
the games, *all* search matches can be found.

It is also quite large - increasing the ScidvsPC binary by something like
25%, and noticeably increasing compile time too.
So i guess we'll have to consider making this a configure-time option. I'm
not sure though.

Anyway - just a heads up about how it is going.

Steven

On Thu, Nov 16, 2017 at 1:10 PM, Lionel <hay...@gm...> wrote:

>
> I've ported Lewis Stiller's and Gady Costeff's CQL v5.1 implementation (
> http://www.gadycosteff.com/cql) to Scid vs PC as a fully integrated
> search engine.
>
> I and a couple of others have been beating up on the engine for 10-12 days
> and it appears to be quite solid. The port was relatively trivial, so I'm
> not expecting many problems, but I'm looking for more volunteers to help
> test what is a rather extensive grammar, especially on non-Linux platforms.
>
> In particular, it would be very helpful if someone possessing Harold van
> der Heijden's study database (http://hhdbv.nl/) could test the port
> against that db. There is a set of scripts in the examples directory to aid
> in that endeavor. I've done my best to find or generate games which test
> the edge cases which are not likely to be present in one's typical chess
> database, but Stiller recommends Heijden's db as something of a testing
> high bar.
>
> The build process for including the CQL engine in the Scid executable has
> only been implemented for the Linux platform, as I have no way of testing
> on other platforms.  I will happily incorporate patches for the Makefiles
> for other platforms as they are offered.  The build requires a C++11
> compiler. The CQL engine has been successfully compiled with GNU g++ and
> with clang, as well as with Microsoft Visual C++.  The entire Scid+CQL
> build has been tested with GNU g++.
>
> I very strongly recommend that users of this feature read the CQL Search
> section in the Help Guide before getting started. There are some essential
> guidelines in that section which can save one from any number of major
> headaches and points of confusion.  Read all of it.  Twice.
>
> Please contact me off list for the [rather large] CQL patch, which applies
> cleanly to the v4.18.1 tarball.  To avoid spamming this list with
> CQL-related noise, please address all questions/problems directly to me.
> Further notices, clarifications, faqs, update patches, etc., coming from me
> will be addressed only to those who have requested the patch.
>
> I'll be welcoming comments and feedback and suggestions while I'm
> eliminating the last of the minor memory leaks in the engine over the next
> few weeks, and hope to release a final patch toward the end of the year.  I
> personally find this integrated CQL engine an order-of-magnitude more
> satisfying to use than the standalone version, and the language, itself,
> continues to impress the hell out of me.
>
>
> Many thanks go to Stiller and Costeff for their innovative work on CQL,
> and to Steve Atkinson for his long years of dedication to the Scid vs PC
> project.
>
>
> -- Lionel
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Scidvspc-users mailing list
> Sci...@li...
> https://lists.sourceforge.net/lists/listinfo/scidvspc-users
>

Re: [Scidvspc-users] CQL port to Scid vs PC

[Lionel <hay...@gm...>]

On 11/20/2017 02:03 AM, Steve A wrote:
> I might just mention a few things. This is CQL 5.1. It is not really
> compatible with CQL 3 , nor (sadly) with Scidb's new implementation.. so
> keep this in mind when looking for example scripts.

We understand that there are many CQL users out there still using v3.x, 
since that's the version found in Chess Assistant.  And those users will 
probably like to be able to continue using v3.x, since the switch-over 
to v5.1 (from what I've seen) in non-trivial.

It might be practical to introduce a v3.x compatibility layer on top of 
v5.1 -- much as Gregor is intending to do with Scidb.  But I would have 
to get the 3.x parser from S&C and maybe some other retired code of 
theirs, and this would further expand the size of the executable.

In any event, this is way down on the list of priorities.


-- Lionel