New shell scripts for testing, used for travis CI (#957)
Perform more complex verification and validation.
- signature.sh : extract and optionally compare Bench/Signature/Node count.
- perft.sh : verify perft counts for a number of positions.
- instrumented.sh : run a few commands or uci sequences through valgrind/sanitizer instrumented binaries.
- reprosearch.sh : verify reproducibility of search.
These script can be used from directly from the command line in the src directory.
Marco Costalba [Sat, 7 Jan 2017 09:13:49 +0000 (10:13 +0100)]
Drop Stats c'tors
Now taht we correctly value-initialize Thread objects,
we don't need c'tors anymore because tables will be
zero-initialized by the compier when Thread object
is instanced.
It can be used uninitialized in time management.
Fixes all valgrind errors on './stockfish go wtime 8000 btime 8000 winc 500 binc 500'
This is one (of the many) quirks of C++. There is a subtle difference between:
new Foo
new Foo()
The first statement calls the default constructor (that in case of a POD leaves data members
uninitialized), the second one performs a value-initialization (that in case of POD is
equivalent to a zero-initialization)
Aram Tumanian [Wed, 21 Dec 2016 11:40:55 +0000 (13:40 +0200)]
Don't clear EasyMove in search()
EasyMove is cleared after every iteration of the
search if the 3rd move in the PV of the main thread
changes from the previous iteration. Therefore,
clearing EasyMove during a search iteration may be
excessive. The tests show that this is indeed unnecessary.
In the new version the EasyMove variable is used only in
the Thread::search function.
Sergei Antonov [Mon, 12 Dec 2016 15:04:16 +0000 (16:04 +0100)]
Threefold repetition detection
Implement a threefold repetition detection. Below are the examples of
problems fixed by this change.
Loosing move in a drawn position.
position fen 8/k7/3p4/p2P1p2/P2P1P2/8/8/K7 w - - 0 1 moves a1a2 a7a8 a2a1
The old code suggested a loosing move "bestmove a8a7", the new code suggests "bestmove a8b7" leading to a draw.
Incorrect evaluation (happened in a real game in TCEC Season 9).
position fen 4rbkr/1q3pp1/b3pn2/7p/1pN5/1P1BBP1P/P1R2QP1/3R2K1 w - - 5 31 moves e3d4 h8h6 d4e3
The old code evaluated it as "cp 0", the new code evaluation is around "cp -50" which is adequate.
Patch has been rewritten into current form for simplification and
logic slightly changed so that return a draw score if the position
repeats once earlier but after or at the root, or repeats twice
strictly before the root. In its original form, repetition at root
was not returned as an immediate draw.
After retestimng testing both version with SPRT[-3, 1], both passed
succesfully, but this version was chosen becuase more natural. There is
an argument about MultiPV in which an extended draw at root may be sensible.
See discussion here:
Alain SAVARD [Wed, 28 Dec 2016 22:14:09 +0000 (17:14 -0500)]
Small eval cleanup and renaming
Non-functional changes
a) splitting the threat array to avoid using an enum
b) reorder the scores according to functions where they are used.
c) declarations in evaluate_pieces after the const(s) like elsewhere
d) more compact definitions of KingFlank,
now that we need it also for the PanwLessFlank penalty.
e) reuse CenterFiles in evaluate_space
f) move one line inside next popcount
Explicitly use alpha+1 for beta in NonPV search (#939)
Fixes the only exception, in razoring.
The code already does assert(PvNode || (alpha == beta - 1)), and it can be verified by studying the program flow that this is indeed the case, also for the modified line.
This patch tweaks some pawn values to favor flank attacks.
The first part of the patch increases the midgame psqt values of external pawns to launch more attacks (credits to user GuardianRM for this idea), while the second part increases the endgame connection values for pawns on upper ranks.
Andrey Neporada [Sat, 3 Dec 2016 08:37:07 +0000 (12:37 +0400)]
Help GCC to optimize msb() to single instruction
GCC compiles builtin_clzll to “63 ^ BSR”. BSR is processor instruction "Bit Scan Reverse".
So old msb() function is basically 63 - 63 ^ BSR.
Unfortunately, GCC fails to simplify this expression.
Marco Costalba [Fri, 25 Nov 2016 15:51:24 +0000 (16:51 +0100)]
Fix compile under Windows XP
The needed Windows API for processor groups could be missed from old Windows
versions, so instead of calling them directly (forcing the linker to resolve
the calls at compile time), try to load them at runtime. To do this we need
first to define the corresponding function pointers.
Also don't interfere with running fishtest on numa hardware with Windows.
Avoid all stockfish one-threaded processes will run on the same node
Aram Tumanian [Mon, 21 Nov 2016 15:17:47 +0000 (17:17 +0200)]
Fix the pawn hash failure when the pawn key is 0
This patch fixed bugs #859 and #882.
At initialization we generate a new random key (Zobrist::noPawns).
It's added to the pawn key of all positions, so that the pawn key
of a pawnless position is no longer 0.
Marco Costalba [Tue, 22 Nov 2016 06:41:46 +0000 (07:41 +0100)]
Handle Windows Processors Groups
Under Windows it is not possible for a process to run on more than one
logical processor group. This usually means to be limited to use max 64
cores. To overcome this, some special platform specific API should be
called to set group affinity for each thread. Original code from Texel by
Peter Österlund.
Tested by Jean-Paul Vael on a Xeon E7-8890 v4 with 88 threads and confimed
speed up between 44 and 88 threads is about 30%, as expected.
Makes the actual number of nodes searched match closely
the number of nodes requested, by increasing the frequency
of checking the number of nodes searched at low node count.
All other searches retain the default checking frequency of
once per 4096 nodes, and are thus unaffected.
Aram Tumanian [Fri, 11 Nov 2016 13:02:28 +0000 (15:02 +0200)]
Make a version of Position::do_move() without the givesCheck parameter
In 10 of 12 calls total to Position::do_move()the givesCheck argument is
simply gives_check(m). So it's reasonable to make an overload without
this parameter, which wraps the existing version.
joergoster [Wed, 9 Nov 2016 15:42:27 +0000 (16:42 +0100)]
FEN parsing: add a second check for correctly setting e.p. square
Currently, we only check if there is a pawn in place
to make the en-passant capture. Now also check that
there is a pawn that could just have advanced two
squares. Also update the corresponding comment.
This makes the parsing of FENs a bit more robust, and
now correctly handles positions like the one reported by Dann Corbit.
position fen rnbqkb1r/ppp3pp/3p1n2/3P4/8/2P5/PP3PPP/RNBQKB1R w KQkq e6
d
+---+---+---+---+---+---+---+---+
| r | n | b | q | k | b | | r |
+---+---+---+---+---+---+---+---+
| p | p | p | | | | p | p |
+---+---+---+---+---+---+---+---+
| | | | p | | n | | |
+---+---+---+---+---+---+---+---+
| | | | P | | | | |
+---+---+---+---+---+---+---+---+
| | | | | | | | |
+---+---+---+---+---+---+---+---+
| | | P | | | | | |
+---+---+---+---+---+---+---+---+
| P | P | | | | P | P | P |
+---+---+---+---+---+---+---+---+
| R | N | B | Q | K | B | | R |
+---+---+---+---+---+---+---+---+
Fen: rnbqkb1r/ppp3pp/3p1n2/3P4/8/2P5/PP3PPP/RNBQKB1R w KQkq - 0 1
Fix undefined behaviour with unaligned loads in syzygy code
Casting a pointer to a different type with stricter alignment
requirements yields to implementation dependent behaviour.
Practicaly everything is fine for common platforms because the
CPU/OS/compiler will generate correct code, but anyhow it is
better to be safe than sorry.
Testing with dbg_hit_on() shows that the unalignment accesses are
very rare (below 0.1%) so it makes sense to split the code in a
fast path for the common case and a slower path as a fallback.
Small fixes for compilation with sanitize=yes optimize=no,
by always adding -fsanitize=undefined to the LDFLAGS as required.
Updates config-sanity to check&report the status of the flag.
Marco Costalba [Sat, 16 Jul 2016 06:10:45 +0000 (08:10 +0200)]
Rewrite syzygy in C++
Rewrite the code in SF style, simplify and
document it.
Code is now much clear and bug free (no mem-leaks and
other small issues) and is also smaller (more than
600 lines of code removed).
All the code has been rewritten but root_probe() and
root_probe_wdl() that are completely misplaced and should
be retired altogheter. For now just leave them in the
original version.
Code is fully and deeply tested for equivalency both in
functionality and in speed with hundreds of games and
test positions and is guaranteed to be 100% equivalent
to the original.
Tested with tb_dbg branch for functional equivalency on
more than 12M positions.
stockfish.exe bench 128 1 16 syzygy.epd
Position: 2016/2016
Total 12121156 Hits 0 hit rate (%) 0
Total time (ms) : 4417851
Nodes searched : 1100151204
Nodes/second : 249024
Tested with 5,000 games match against master, 1 Thread,
128 MB Hash each, tc 40+0.4, which is almost equivalent
to LTC in Fishtest on this machine. 3-, 4- and 5-men syzygy
bases on SSD, 12-moves opening book to emphasize mid- and endgame.
Score of SF-SyzygyC++ vs SF-Master: 633 - 617 - 3750 [0.502] 5000
ELO difference: 1
Avoid shifting negative signed integers and use typed
enum to avoids decrementing a variable beyond its defined
range, like:
for (Rank r = RANK_8; r >= RANK_1; --r)
Changes were tested individually and passed SPRT[-3, 1].
syzygy [Wed, 26 Oct 2016 21:01:11 +0000 (23:01 +0200)]
Output PV if last iteration does not complete
Instead of outputting "info nodes ... time ..." when the last
iteration is interrupted, simply call UCI::pv() to output the PV.
I thought about calling UCI:pv() with bounds -VALUE_INFINITE, VALUE_INFINITE
to avoid "lowerbound" or "upperbound" appearing in it, but I'm not sure that
would be any better.
This patch fixes rare inconsistencies between the first move of
the last PV output and the bestmove played. It also makes sure
that all the latest statistics are sent to the GUI (not only nodes
and time but also nps, tbhits, hashfull).
ajithcj [Sat, 15 Oct 2016 07:29:24 +0000 (07:29 +0000)]
Remove useless assignments to currentMove
We reference (ss-1)->currentMove, i.e. we peek
current move of the parent node, so currentMove
should be valid in the main move loop, when we
search() the subtree, but outside of main loop
it is useless.
Jacques [Sat, 8 Oct 2016 10:15:43 +0000 (12:15 +0200)]
Fixes for ARM compilation: take 2
The target:
Odroid U3 (http://www.hardkernel.com/main/products/prdt_info.php?g_code=g138745696275)
Debian Jessie
As listed in #550 and #638 three modifications are needed for compilation to work:
float-abi flag for GCC If an FPU is present and supported by the installed os then passed value need to be hard.
I didn't find any better solution than using readelf to check for the availibilty of Tag_ABI_VFP_args which sould indicate support for the FPU. The check is only done if the arch is arm and if readelf is not present
on the system, there will be an error (/bin/sh: 1: readelf: not found) but it will not break and will continue with the default softfp value. Outputing the error is not really acceptable but I wanted some feedback on the
check itself.
-lpthread is needed on armv7 outside of Android
I replaced UNAME with KERNEL and OS to allow to differentiate Android.
m32 flag
My understanding is that outside of Android the flag is generating errors on armv7.
These modifications should introduce change only for non Android armv7 build.
Jacques [Sat, 8 Oct 2016 10:15:43 +0000 (12:15 +0200)]
Fixes for ARM compilation
The target:
Odroid U3 (http://www.hardkernel.com/main/products/prdt_info.php?g_code=g138745696275)
Debian Jessie
As listed in #550 and #638 three modifications are needed for compilation to work:
float-abi flag for GCC If an FPU is present and supported by the installed os then passed value need to be hard.
I didn't find any better solution than using readelf to check for the availibilty of Tag_ABI_VFP_args which sould indicate support for the FPU. The check is only done if the arch is arm and if readelf is not present
on the system, there will be an error (/bin/sh: 1: readelf: not found) but it will not break and will continue with the default softfp value. Outputing the error is not really acceptable but I wanted some feedback on the
check itself.
-lpthread is needed on armv7 outside of Android
I replaced UNAME with KERNEL and OS to allow to differentiate Android.
m32 flag
My understanding is that outside of Android the flag is generating errors on armv7.
These modifications should introduce change only for non Android armv7 build.
atumanian [Thu, 6 Oct 2016 17:55:10 +0000 (20:55 +0300)]
Optimisation of Position::see and Position::see_sign
Stephane's patch removes the only usage of Position::see, where the
returned value isn't immediately compared with a value. So I replaced
this function by its optimised and more specific version see_ge. This
function also supersedes the function Position::see_sign.
bool Position::see_ge(Move m, Value v) const;
This function tests if the SEE of a move is greater or equal than a
given value. We use forward iteration on captures instread of backward
one, therefore we don't need the swapList array. Also we stop as soon
as we have enough information to obtain the result, avoiding unnecessary
calls to the min_attacker function.
Speed tests (Windows 7), 20 runs for each engine:
Test engine: mean 866648, st. dev. 5964
Base engine: mean 846751, st. dev. 22846
Speedup: 1.023
VoyagerOne [Mon, 3 Oct 2016 13:36:40 +0000 (09:36 -0400)]
Allow inCheck pruning
This is a bit tricky because we don't want
to prune the only legal evasions, even if
with negative SEE. So add an assert to avoid
this subtle bug to slip in later.