#1 Britton's chess960 patch

[Steven]

Hey,

I wrote a patch for chess960. It works pretty well I think. If importing pgn make sure that castling in the FEN string is like KQkq instead of the other ways ive seen it done.

This code knows no difference between chess960 castling and regular chess castling, it is all chess960 castling. Since regular chess is just one position in chess960, it works for standard chess as well.

For now, to castle in the gui, the king must capture a friendly rook. The move 2 spaces is not implemented anymore, even for regular chess. I tried several locations to do it, but failed mainly from laziness. I think the gui should substitute a friendly rook capture for a king moving 2 as early as possible before passing move to the c code. I kind of want to keep that out of there to keep it simple.

FICS has not been touched nor has any tcl code

The global encodeVariation func is now a member of Game. The encoding process needs current positions as it encodes kings so it knows what the king is actually trying to do, so importing large pgns is maybe slower now!

Also, Position class now contains a pointer to the Game that owns it, so a position can know about the start position. Some position's Game * is NULL (like opening books, etc, i think).

Let me know what you think!

I would like to continue on. Maybe the FICS functionality could be improved. Maybe bughouse or crazyhouse. I do not know tcl/tk, although that may change.

Im going to try to attach to this email a pgn of some fics chess960 games for to play with, import, etc

Thanks,
britton

[Steven]

Thanks Britton.

There is a bug with several variations and game saves.

[FEN "bnrbkrqn/pppppppp/8/8/8/8/PPPPPPPP/BNRBKRQN w KQkq - 0 1"]

{SP 353} 
1. c4 c5 2. Ng3 e6 3. e3 b6 4. Nc3 Nc6 5. Nge4 Be7 
    ( 5. ... Bf6 6. Ng3 Bxc3 7. bxc3 )
6. Nb5 O-O-O 7. Be2 Rfe8 8. O-O-O 1/2-1/2

Then , add this variation 7. Bf3
Save game. Double click on game to reload it. Change has disappeared.

{SP 353} 
1. c4 c5 2. Ng3 e6 3. e3 b6 4. Nc3 Nc6 5. Nge4 Be7 
    ( 5. ... Bf6 6. Ng3 Bxc3 7. bxc3 )
6. Nb5 O-O-O 7. Be2 
    ( 7. Bf3 )
7. ... Rfe8 8. O-O-O 1/2-1/2

[brit tonf]

im seeing it.. This one might take some wrestling

[Steven]

Attached is a tcl patch to
- Hack to allow old style castling
- disable board setup KQkq checks
- Enable fics castling (in normal chess)
- Send chess960 notice to uci engines.

[brit tonf]

Here is a patch to fix the variation disappearance bug (and another related that i found during the process of fixing). This patch is for after the first main 960 patch has been applied.

game.h header file changed so you guys should do a "sudo make clean" and then recompile the whole thing to be safe or you might see a segfault.

I removed Game::EncodeVariation and turned it back into static func encodeVariation like it was originally. Turns out I was overthinking a little due to not understanding enough, although the func is modified to accept a Position ptr still. This ptr is created just for iterating during encode process and then is deleted.

[Steven]

Thanks... I'll give it a go shortly.
Just now - I've been playing a few games and saving the results to a database
Bad things are definitely happening... games getting truncated and/or non-standard starts
appearing in games.

(Re the format of diffs... I find it easiest just to send diffs against svn.
Just send "svn diff src > patch". When i recieve it, I will revert back to subversion , then patch with each new patch.)

[brit tonf]

Hopefully the variation_disappearance_fix.diff will fix everything. Let me know if you can still reproduce any strangeness anywhere dealing with saving/opening games with or without variations after applying the patch.

[brit tonf]

The eco file reading was failing, but this is now fixed. Positions created during the creation of the opening book when reading the eco file do not have a Game ptr for owner (or rather it is NULL), so therefore it was failing to figure out a legal castle due to not knowing where rooks started from. So the GetRookOrigSq func returns hardcoded A1,H1,etc of standard start if the pos has a NULL owner.

For this patch, go into src and apply it to the latest already patched position.cpp.

Just to double check, in the future when giving patches are you saying you want diffs from -r2569 root folder and then you will go back to -r2569 and then apply it?

[Gregor Cramer]

This patch (for board setup) is not yet complete. Consider this FEN: "5rkr/8/8/8/8/8/8/R1R1K3 w Q - 0 1" (the start position is not known in this case). The pecularity of this FEN is the ambuigity of the castle rook. That's why the X-FEN has been defined. If the rook on A file is the castle rook, then the correct FEN is "5rkr/8/8/8/8/8/8/R1R1K3 w A - 0 1", which resolves the ambuigity (see X-FEN spec). So the board setup needs additional combo-boxes, providing the selection of the castle rook files (white and black).

Probably also the Shredder FEN should be recognized (see Wikipedia). I've attached a TCL file which provides conversion functions between Shredder and X-FEN (tested). Because X-FEN is standard, Scid should save the X-FEN.

[brit tonf]

I will have to learn more about X-FEN. I just read the wikipedia article, but still need to research a little on some details. I notice that lichess fens for chess960 games have KQkq in their fens no matter where the rooks are, but Im guessing that is ok if it is a regualr game with no ambiguity, starting at the beginning, and then when you export mid game, maybe they tuen the fen into BGbg for example. Looks like it shouldn't be too difficult once i grasp the details. Looks very similar to regular fen, so hopefully just some tweaks. Also it seems that KQkq is still interpreted as valid in xfen if the rooks are on A or H.

It seems like the bulk of the work might have to be done in the tcl interface, which I won't really be able to touch yet, due to total inexperience with tcl. I noticed that the dropdown box for castling rights does not have all the possible permutations even for regular fen at this time (like Kq and Qk and KQk and Kkq ,etc, etc are missing) Not really sure a dropdown is a good solution anyway since it would technically need 16 items for just k and q style. If you add B-G files the dropdown box would be ridiculous. Seems like the best way to do it would be to get rid of the dropdown and supply only a text box where you can just type in KQk or BGg or Gc or whatever you want.

I can modifythe c code after I strategize and learn more about this. It actually doesn't seem like too much work (just jinxed myself) if all that needs to be worried about is an xfen string being passed in and creating an xfen string. Hopefully just some tweaks.

[brit tonf]

ok i see what you mean about the combo box. It can be a box with values filled only after the pieces are placed so it won't be so giant or prone to error.

[Gregor Cramer]

I notice that lichess fens for chess960 games have KQkq
in their fens no matter where the rooks are

The X-FEN is downward compatible, the standard start position is the same in X-FEN, because the rook positions are not ambiguous. The difference is with ambiguous rook positions. In this case X-FEN can be ugly, because it might contain a mix - like in this example: "rn2k1r1/ppp1pp1p/3p2p1/5bn1/P7/2N2B2/1PPPPP2/2BNK1RR w Gkq - 4 11". That's why the Shredder-FEN is also popular, with Shredder-FEN the same position will be expressed as "rn2k1r1/ppp1pp1p/3p2p1/5bn1/P7/2N2B2/1PPPPP2/2BNK1RR w Gga - 4 11". The Shredder-FEN refrains from compatiblity, it only uses the rook files. So I think it would be good if the box can display both, depending on user's choice, with the provided conversion functions this shouldn't be a bigger effort.

ok i see what you mean about the combo box. It can be a box
with values filled only after the pieces are placed so it
won't be so giant or prone to error.

In fact I've meant four combo-boxes, for black/white, and queenside/kingside, allowing the selection of the rook file.

[Gregor Cramer]

Probably the board setup provides the possibility to enter the Chess960 position number. I've attached an extended implementation of chess960.tcl which contains a mapping function from position number to FEN.

[Steven]

Thanks for that Gregor. I will implement chess960 in the setup board dialog sometime.

Britton - yes , eco seems to load now.

I have added a patch to svn - at least for the time being - patches/chess960-new.patch.
to show what the latest code i have is.

Also - i added 960 support for UCI computer game. Playing stockfish
works i think. Just enable start-from-current position.
Playing chess960 (wild) on fics looks a little more complicated... but maybe i will do it this weekend.

[brit tonf]

yeah i played stockfish last night that way a couple of times. (t won)

I saw an email about fics truncating games, but cannot find the post. Let me know if that is still happening! If possible to somehow save the pgn or something to know the setup and move order before saving game, that would be helpful. I noticed that when encoding had bugs, some pgns expose it and some don't, pretty consistently. As in, it is likely because of some move order and/or setup instead of fics specifically.

All that being said, I hope and think that the patch above (variation_disappearance_fix.diff) has fixed everything with saving (encoding) games. It was only when I wrote that patch that I finally understood all the issues going into giving the recursive function encodeVariation the info it needs during arbitrary recursions. Up to then i was basically like, "i dunno, lets try this, this seems like it might work" and then it would fix something and then break something else.

I still need to familiaraize myself with fics. In telnet i do i think "games wild fr" just to look at some games and then when i watch them, they are all regular chess setups. I will find time soon to fool around with it and hopefully play some games.

[brit tonf]

Gregor the start position script looks good. It matches with the start position numbering scheme that seems to be adopted most places.
One thing though (and i may have done this tcl wrong or still not understanding xfen correctly):
I put this line at the end of your script:

puts [::chess960::convertToXFEN [::chess960::numberToFEN 735]]

and then ran the script and it spit out:

rknqnrbb/pppppppp/8/8/8/8/PPPPPPPP/RKNQNRBB w KQkq - 0 1

Should not the castling be FQfq ? (queenside rook in normal start so Qq, kinside rook on f file so Ff)

[Gregor Cramer]

One thing though (and i may have done this tcl wrong or still not
understanding xfen correctly): I put this line at the end of your script:
~~~
puts [::chess960::convertToXFEN [::chess960::numberToFEN 735]]
~~~
and then ran the script and it spit out:
~~~
rknqnrbb/pppppppp/8/8/8/8/PPPPPPPP/RKNQNRBB w KQkq - 0 1
~~~
Should not the castling be FQfq ? (queenside rook in normal start so Qq,
kinside rook on f file so Ff)

The general rule of X-FEN is: if the castling rook is the outermost rook, then
the castling direction will be specified (one of K,Q,k,q). If the castling rook
is the innermost rook, and additionally an outermost rook exists (for example
Ke8, Rf8, Rh8 - Rf8 is innermost, Rh8 is outermost), then the rook file will be
specified. This rule is compatible to standard chess. Any Chess960 start
position contains the string "KQkq ", because the outermost rook is the
castling rook. The X-FEN is of course confusing, and one of the reasons why
many people are preferring the Shredder-FEN, here always the rook files will be
specified: "rknqnrbb/pppppppp/8/8/8/8/PPPPPPPP/RKNQNRBB w FAfa - 0 1".

[Gregor Cramer]

About the variation bug:

I strongly recommend not to change function Game::encodeVariation(). The chess 960 castle moves cannot have an impact on this algorithm. If I have a closer look on Steve's example

6. Nb5 O-O-O 7. Be2 
    ( 7. Bf3 )
7. ... Rfe8 8. O-O-O

with the debugger I can see that MoveForward() gives an error with 6...O-O-O, so there is something wrong with Position::DoSimpleMove() related to Chess960 castling.

[brit tonf]

Ok i understand now about the castle string. Thanks for the explanation.

MoveForward had been taken out of encodeVariation earlier. There is a patch above (variation_disappearance_fix.diff) that has changes to encodeVariation. If you are still having trouble after applying that patch let me know. It seems you are seeing the error that happens before it is applied. I may post a giant sanity patch pretty soon since there have been a lot of patches, patches on top of patches, etc in this thread.

encodeVariation had to be changed in order to contain a position ptr used so the function encodeKing can know if a castle is being attempted (since a king simply moving 1 square does not have enough information anymore to determine if a castle is occurring or not). It will still encode 9 or 10 for a king move castle (see encodeKing in game.cpp) so therefore no saved game information will be any different than before. The encodeVariation func had changed a lot more than it is now (with bugs), and has since been unchanged to only have a ptr and a call to DoSimpleMove for this ptr (instead of MoveForward, which doesn't do what is needed). This ptr is temporary and is created and destroyed in the function Game::Encode that calls encodeVariation.

I will post a sanity patch here in a minute

[brit tonf]

Actually i just noticed steve committed a good sanity patch into -r2571 that we can base off of for awhile. (Thanks for the creds steve. First time contributing to an official open source project, pretty cool!)

I realized I forgot to post a diff that isn't in that patch, or maybe it got deleted, or maybe i put it in the mailing list. It grants castling of all types when setting up from FEN and only when the FEN stops right after the whose move letter (like it is not specified). Without the diff it will not grant if king or rook not in classical spots. Sledgehammer approach for now. Will likely change whenever interface for setting up board and xfen stuff get going. Probably not really an important diff but i will post anyway.

Also I saw a thing in the committed patchfile whose diff was not posted here in the thread i dont think, probably done by steve. In tcl/tools/sergame.tcl something about UCI chess960

[Steven]

Without the diff it will not grant if king or rook not in classical spots.

Yes - i don't think we need this diff and i didnt include it. The change i made here

proc validateFEN {fen} {
global setupFen
+# Don't worry about validating castling in chess960
+ return $fen

Should allow any KQkq right regardless of king posiiton.
Please check your gui that this change is included. Am i missing something ?

[Steven]

I have added the Chess960 position generator to the setup board widget,
and also changed program name to ScidvsPC960.
to test, just do a svn revert of local changes and

svn update && patch -p0 <patches/chess960-new.patch && make clean install

[Steven]

I saw an email about fics truncating games, but cannot find the post. Let me know if that is still happening!

It might just be something unusual happening with FICs.
At the end of a game, it is sometimes deciding game position is erroneous, and rereading the game or something.

[brit tonf]

Here is a patch to export a fen that disambiguates rook positions as described above by gregor (i think). This is just export. Importing not done and still might pick the wrong rook if 2 rooks on same side of king. Tried last night for it, but it turned into a huge morass. I can see where shredder would be a thousand times easier to program for than xfen.

Additionally, this patch also fixes a bug that has not reared its head yet in GetRookOrigSq if it would ever be called trying to get original rook sq of the opposite team for whatever reason.

Yes - i don't think we need this diff and i didnt include it. The change i made here....
Should allow any KQkq right regardless of king posiiton.
Please check your gui that this change is included. Am i missing something ?

You aren't really missing anything. It is unimportant and was incomplete and is on the verge of being obsolete anyway. If you tried to import fen like "nrqnbkrb/pppppppp/8/8/8/8/PPPPPPPP/NRQNBKRB w" with no castling info, the old code would look for A1,A8,H1,H8 to assume you want castling and give it to you if those squares have rooks. The patch i did that was not included just gave you castling for short fens like that no matter what, but let's just leave it out like you did. It will become obsolete whenever i can get xfen import to work for the midgame setups (it is painful)

[Steven]

OK, i'll add this code.
Yes... Handling castling info isn't quite complete.. but we'll get there. :)

[brit tonf]

I have not had a chance to keep working on the xfen import. Hopefully tonight, but maybe read below and tell me what you think. I got it to a point a day or 2 ago where it is probably mostly done.

Shredder might should be considered for all backend operations, xfen is nuts to handle, especially in the backend guts of the program, causing the c code to keep track of a lot of things it might not have needed to before (or at least look it up), especially for this import. Export was easy by comparison for some reason.

Shredder is much more sane. Xfen is harebrained by comparison, although i know why it exists the way it does, so im not really complaining. It occurred to me it might be easier (and saner) to have the tcl do compatibility for fen/xfen/shredder in the interface only (which is the only place compatibility is really needed) and send only shredder fens to and from the c code. It seems tcl being a scripting language it would have string funcs to quickly translate K's to G's for example by simply reading the positional part of the fen string right on the spot, if needed, for the end users preference. For an export, the c code could return the position portion of the start fen to a tcl command if it doesn't already have access to it.

Let me know how that sounds. If you guys don't like the idea, I have the xfen import most of the way done.