In the current master, ThreatByKing is an array of two Scores, one for
when we have a single attack and one for when we have many. The latter
case is very rarely called during bench and was recently given a strange
negative value during a tuning run, as pointed out by @candirufish on
commit efd4ca2. Here, we simplify away this second case entirely, and
increase the remaining ThreatByKing to compensate.
Although I derived the parameter tweak independently, with the goal of
preserving the same average bonus, I later noticed that a very similar
Score had already been derived by an ongoing SPSA tuning session.
I therefore recognize @candirufish for first discovering these values.
I would also like to thank @Rocky640 for valuable feedback that pointed
me in the direction of ThreatByKing.
Ondrej Mosnáček [Sat, 9 Jun 2018 11:45:49 +0000 (13:45 +0200)]
Move PSQ score to Position
This patch simplifies Position::do_move() by moving the PSQ score from
StateInfo to Position and updating it inside the put/remove/move_piece
functions.
The downside is that there is now slightly more computation done in
Position::undo_move(), but the fishtest results are Elo neutral.
protonspring [Sun, 17 Jun 2018 02:26:25 +0000 (20:26 -0600)]
Remove make_bitboard()
In current master, the function make_bitboard() does nothing apart from
helping initialize the SquareBB[] array. This seems like an unnecessary
abstraction layer.
The advantage of make_bitboard() is we can define a bitboard, in a simple
and general way, not only from a single square but also from a list of
squares. It is more elegant, faster and readable than combining multiple
SquareBB explicitly, but the last complex use case in evaluation was
simplified away a few months ago.
If make_bitboard() becomes useful again to define complicated bitboards,
it will be easy enough to reintroduce it using this pull request as
an implementation reference.
Alain SAVARD [Sun, 24 Jun 2018 22:06:13 +0000 (18:06 -0400)]
Simplify HinderPassedPawn bonus
Make sure each piece is not scored more than once as a passed pawn "hinderer",
by scoring only the blockers along the passed pawn path. Inspired by TCEC Game 29.
Passed STC as a simplification
http://tests.stockfishchess.org/tests/view/5b3016d00ebc5902b2e58552
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 75388 W: 16656 L: 16641 D: 42091
Passed LTC as a simplification
http://tests.stockfishchess.org/tests/view/5b302ed90ebc5902b2e587fc
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 49157 W: 8460 L: 8386 D: 32311
Current master was also counting the number of attacks along a passed pawn path,
which might be misleading:
a) a defender might be counted many times for the same pawn path. For example a
White rook on a1 attacking a black pawn on a7 would score the bonus * 6 but
would be probably better placed on a8
b) a defender might be counted on different pawn paths and might be overloaded. For
example a Ke4 or Qe4 against pawns on d6 and f6 would score the bonus * 6.
Counting each blocker or attacker only once is more complicated, and does not help
either: http://tests.stockfishchess.org/tests/view/5b2ff1cb0ebc5902b2e582b2
After this small simplification, there might be ways to increase the HinderPassedPawn
penalty.
Give more incentive to king activity in the endgame by increasing the weight
of the "outflanking" variable from 8 to 12 in the function evaluate_initiative().
Michael An [Thu, 21 Jun 2018 01:05:27 +0000 (21:05 -0400)]
Fix GCC 8 cast warnings
Silences the following warnings when compiling with GCC 8.
The fix is to use an intermediate pointer to anonymous function:
```
misc.cpp: In function 'int WinProcGroup::get_group(size_t)':
misc.cpp:241:77: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun1_t' {aka 'bool (*)(_LOGICAL_PROCESSOR_RELATIONSHIP, _SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX*, long unsigned int*)'} [-Wcast-function-type]
auto fun1 = (fun1_t)GetProcAddress(k32, "GetLogicalProcessorInformationEx");
^
misc.cpp: In function 'void WinProcGroup::bindThisThread(size_t)':
misc.cpp:309:71: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun2_t' {aka 'bool (*)(short unsigned int, _GROUP_AFFINITY*)'} [-Wcast-function-type]
auto fun2 = (fun2_t)GetProcAddress(k32, "GetNumaNodeProcessorMaskEx");
^
misc.cpp:310:67: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'fun3_t' {aka 'bool (*)(void*, const _GROUP_AFFINITY*, _GROUP_AFFINITY*)'} [-Wcast-function-type]
auto fun3 = (fun3_t)GetProcAddress(k32, "SetThreadGroupAffinity");
^
```
DU-jdto [Wed, 13 Jun 2018 05:22:52 +0000 (15:22 +1000)]
Remove lmrDepth restriction on quiet see pruning
And tweak the threshold value. With this threshold and the current piece
values, this permits see pruning on quiets to be done up to an lmrDepth
of 9 (beyond that the threshold is below -QueenValueMg and see_ge will
pass unconditionally).
protonspring [Mon, 11 Jun 2018 08:17:03 +0000 (10:17 +0200)]
Optimize an expression in endgame.cpp
I believe using foward_file_bb() here is fewer instructions.
a) Fewer instructions and probably more clear (debatable).
b) Possible that a lookup is slower than a few local operations, but the
forward_file_bb table is probably used often enough that it is always
cached.
candirufish [Sun, 10 Jun 2018 08:53:05 +0000 (01:53 -0700)]
Simplify capture pruning margin formula
Using just `PawnValueEg * depth` as Capture Prune Margin. There was a bunch
of patches that passed recently regarding captures, maybe this part of the
master code redundant? The patch was tested as a simplification:
After several tests it seems best to increase contempt from 12 to 21. This does
not regress against contempt=0 and gives a gain of around 7-8 elo against SF 7
in comparison to current default contempt.
STC: Test for non-regression contempt=21 against contempt=0
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 71250 W: 13956 L: 13926 D: 43368
http://tests.stockfishchess.org/tests/view/5b19a58d0ebc5902ab9c3bfa
STC: Test contempt 21 against SF 7
ELO: 190.06 +-2.8 (95%) LOS: 100.0%
Total: 40000 W: 22608 L: 2676 D: 14716
http://tests.stockfishchess.org/tests/view/5b19a6520ebc5902ab9c3c0e
STC: Test master against SF 7 for comparison
ELO: 182.95 +-2.7 (95%) LOS: 100.0%
Total: 40000 W: 21905 L: 2595 D: 15500
http://tests.stockfishchess.org/tests/view/5b16f5bc0ebc59214346d5ca
LTC: Test for non-regression contempt=21 against contempt=0
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 47666 W: 6914 L: 6832 D: 33920
http://tests.stockfishchess.org/tests/view/5b1a170b0ebc5902ab9c3fde
LTC: Test contempt 21 against SF 7
ELO: 203.92 +-2.6 (95%) LOS: 100.0%
Total: 40000 W: 22447 L: 1340 D: 16213
http://tests.stockfishchess.org/tests/view/5b1a174b0ebc5902ab9c3fe1
LTC: Test master against SF 7 for comparison
ELO: 196.08 +-2.6 (95%) LOS: 100.0%
Total: 40000 W: 21639 L: 1191 D: 17170
http://tests.stockfishchess.org/tests/view/5b1a17e40ebc5902ab9c3fe4
candirufish [Wed, 6 Jun 2018 08:13:08 +0000 (10:13 +0200)]
Quiet move soft fail high bonus
Extra bonus for quiet move creating a huge soft fail high (triggered
in 21% of quiet bestmoves on a normal bench run). Pb00067 original idea
using PawnValueMg.
This has the property of raising alpha before calling qsearch(), thus
maybe giving some more cuts during qsearch(). The patch is equivalent
to the use of cycle detection inside qsearch() at depth 0, but is in
fact implemented by re-ordering code inside search(), which explains
the [0..4] bounds in the following tests.
STC (interrupted after 124250 games, with LLR=0.87):
http://tests.stockfishchess.org/tests/view/5b1500bd0ebc5902a8b420bf
LLR: 0.87 (-2.94,2.94) [0.00,4.00]
Total: 124250 W: 24973 L: 24470 D: 74807
After a helpful suggestion from AppVeyor support staff, moving the Stockfish
execution from ps to cmd seems to work. Alternative to PR #1624 tested in PR #1637.
Guenther Demetz [Mon, 4 Jun 2018 07:10:30 +0000 (09:10 +0200)]
Remove a superfluous subtrahend
The '- 1' subtrahend was introduced for guarding against null move
search at root, which would be nonsense. But this is actually already
guaranteed by the !PvNode condition. This followed from the discussion
in pull request 1609: https://github.com/official-stockfish/Stockfish/pull/1609
joergoster [Fri, 1 Jun 2018 20:35:23 +0000 (22:35 +0200)]
Bugfix of Position::has_repeated()
The function Position::has_repeated() is used by Tablebases::root_probe()
to determine whether we can rank all winning moves with the same value, or
if we need to strictly rank by dtz in case the position has already been
repeated once, and we are risking to run into the 50-move rule and thus
losing the win (especially critical in some very complicated endgames).
To check whether the current position or one of the previous positions
after the last zeroing move has already been occured once, we start looking
for a repetition of the current position, and if that is not the case, we
step one position back and repeat the check for that position, and so on.
If you now look at how this was done before the new root ranking patch was
merged two months ago, it seems quite obvious that it is a simple oversight:
https://github.com/official-stockfish/Stockfish/commit/108f0da4d7f993732aa2e854b8f3fa8ca6d3b46c
More specifically, after we stepped one position back with
```
stc = stc->previous;
```
we now have to start checking for a repetition with
Makes sure the potential benefit of first touch does not depend on
the order of the UCI commands Threads and Hash, by reallocating the
hash if a Threads is issued. The cost is zeroing the TT once more
than needed. In case the prefered order (first Threads than Hash)
is employed, this amounts to zeroing the default sized TT (16Mb),
which is essentially instantaneous.
Follow up for https://github.com/official-stockfish/Stockfish/pull/1601
where additional data and discussion is available.
Stockfish currently takes a while to clear the TT when using larger hash sizes.
On one machine with 128 GB hash it takes about 50 seconds with a single thread,
allowing it to use all allocated cores brought that time down to 4 seconds on
some Linux systems. The patch was further tested on Windows and refined with
NUMA binding of the hash initializing threads (we refer to pull request #1601
for the complete discussion and the speed measurements).
protonspring [Thu, 24 May 2018 16:46:38 +0000 (18:46 +0200)]
Simplify BlockedByPawn to one dimension
I was able to get this to pass which reduces BlockedByPawn to one dimension
with NO distance from edge offset.
GOOD) It's more simple and may provide additional clarity for further
simplifications. Facilitates migrating unblocked to one dimension as well.
BAD) If there is indeed a distance component to BlockedStorm (may or may
not be the case), this obfuscates this component into ShelterStrength and
UnblockedStorm. This may be more convoluted. Also, it may be more convenient
to have each of the three arrays (ShelterStrength, BlockedStorm, and UnBlocked)
be the same size.
As discussed with @pb00068, the condition to prevent recursive verification
was not completely correct. This patch corrects that condition, and adds an
assert. In the current implementation, recursive verification needs to be
avoided in order not to break the verification of the move closer to the
root (i.e. to not set thisThread->nmp_min_ply to zero prematurely).
This patch is tested as a bug fix, based on and tested against PR #1609 .
Simplifying away all the progressKey stuff gives exactly the same bench,
without any speed impact. Tested for speed against master with two benches
at depth 22 ran in parallel:
**testedpatch**
Total time (ms) : 92350
Nodes searched : 178962949
Nodes/second : 1937877
**master**
Total time (ms) : 92358
Nodes searched : 178962949
Nodes/second : 1937709
We also tested the patch at STC for no-regression with [-3, 1] bounds:
mstembera [Wed, 16 May 2018 21:38:13 +0000 (23:38 +0200)]
Fix MSVC errors in tbprobe.cpp
Default template parameters values and recursive functions do not play well
together. Fix for below errors that showed up after updating to latest MSVC.
````
tbprobe.cpp(1156): error C2672:
'search': no matching overloaded function found
tbprobe.cpp(1198): error C2783:
'Tablebases::WDLScore `anonymous-namespace'::search(Position &,Tablebases::ProbeState *)':
could not deduce template argument for 'CheckZeroingMoves'
Tom Truscott [Wed, 16 May 2018 20:47:41 +0000 (22:47 +0200)]
Use cycle detection to bound search value
A position which has a move which draws by repetition, or which could have
been reached from an earlier position in the game tree, is considered to be
at least a draw for the side to move.
How does the algorithm work in practice? The algorithm is an efficient
method to detect if the side to move has a drawing move, without doing any
move generation, thus possibly giving a cheap cutoffThe most interesting
conditions are both on line 1195:
This uses the position keys as a sort-of Bloom filter, to avoid the expensive
checks which follow. For "upcoming repetition" consider the opening Nf3 Nf6 Ng1.
The XOR of this position's key with the starting position gives their difference,
which can be used to look up black's repeating move (Ng8). But that look-up is
expensive, so line 1195 checks that the white pieces are on their original squares.
This is the subtlest part of the algorithm, but the basic idea in the above game
is there are 4 positions (starting position and the one after each move). An XOR
of the first pair (startpos and after Nf3) gives a key matching Nf3. An XOR of
the second pair (after Nf6 and after Ng1) gives a key matching the move Ng1. But
since the difference in each pair is the location of the white knight those keys
are "identical" (not quite because while there are 4 keys the the side to move
changed 3 times, so the keys differ by Zobrist::side). The loop containing line
1195 does this pair-wise XOR-ing.
Continuing the example, after line 1195 determines that the white pieces are
back where they started we still need to make sure the changes in the black
pieces represents a legal move. This is done by looking up the "moveKey" to
see if it corresponds to possible move, and that there are no pieces blocking
its way. There is the additional complication that, to match the behavior of
is_draw(), if the repetition is not inside the search tree then there must be
an additional repetition in the game history. Since a position can have more
than one upcoming repetition a simple count does not suffice. So there is a
search loop ending on line 1215.
On the other hand, the "no-progress' is the same thing but offset by 1 ply.
I like the concept but think it currently has minimal or negative benefit,
and I'd be happy to remove it if that would get the patch accepted. This
will not, however, save many lines of code.
• The code in search() that checks for cycles has numerous possible variants.
Perhaps the check need could be done in qsearch() too.
• The biggest improvement would be to get "no progress" to be of actual benefit,
and it would be helpful understand why it (probably) isn't. Perhaps there is an
interaction with the transposition table or the (fantastically complex) tree
search. Perhaps this would be hard to fix, but there may be a simple oversight.
VoyagerOne [Mon, 14 May 2018 04:52:16 +0000 (06:52 +0200)]
Update search.cpp
At PvNodes allow bonus for prior counter move that caused a fail low
for depth 1 and 2. Note : I did a speculative LTC on yellow STC patch
since history stats tend to be highly TC sensitive
Use the whole kingRing for pawn attackers instead of only the squares directly
around the king. This tends to give quite a lot more kingAttackersCount, so to
compensate and to avoid raising the king danger too fast we lower the values
in the KingAttackWeights array a little bit.
Credits to user @xoroshiro for the idea of using the kingRing for pawn attackers.
How to continue? It seems that the KingAttackWeights[] array stores values
which are quite Elo-sensitive, yet they have not been tuned with SPSA recently.
There might be easy Elo points to get there.
mstembera [Thu, 10 May 2018 20:49:56 +0000 (13:49 -0700)]
Include all blockers in king danger
Simplification: in king danger, include all blockers and not only pinned
pieces, since blockers enemy pieces can result in discovered checks which
are also bad.
I also incorrectly scheduled STC with [0,5] which it failed.
http://tests.stockfishchess.org/tests/view/5af283c00ebc5968e6523f33
LLR: -2.94 (-2.94,2.94) [0.00,5.00]
Total: 12338 W: 2451 L: 2522 D: 7365
Thanks to @vondele and @Rocky640 for a cleaner version of the patch,
and the following comments!
> Most of the pinned, (or for this pull request, blocking) squares were
> already computed in the unsafeChecks, the only missing squares being:
>
> a) squares attacked by a Queen which are occupied by friendly piece
> or "unsafe". Note that adding such squares never passed SPRT[0,5].
>
> b) squares not in mobilityArea[Us].
>
> There is a strong relationship between the blockers and the unsafeChecks,
> but the bitboard unsafeChecks is still useful when the checker is not
> aligned with the king, and the checking square is occupied by friendly
> piece or is "unsafe". This is always the case for the Knight.
protonspring [Thu, 10 May 2018 13:46:13 +0000 (15:46 +0200)]
Consolidate pawn storm types
Simplification: the Unopposed and Unblocked pawn storm types are mathematically
similar enough to combine with no Elo loss. This reduces the pawn storm types
to BlockedByPawn and UnBlocked.
candirufish [Wed, 9 May 2018 07:18:22 +0000 (09:18 +0200)]
Tuned some pawns and evaluation constants
Tuned values in pawns.cpp and evaluate.cpp after a SPSA session:
419k games 60sec 600nodetime. We have adjusted the PassedRank[]
output of the SPSA session to keep increasing values with rank,
and PassedFile[] output to keep the West <–> East symmetry of
the evaluation.
This patch simplifies the control flow in search(), removing an if
and a goto. A side effect of the patch is that Stockfish is now a
little bit more selective at low depths, because we allow razoring,
futility pruning and probcut pruning after a null move.
Alain SAVARD [Tue, 8 May 2018 09:00:51 +0000 (11:00 +0200)]
Drop the lever condition for backwards
We can view the patch version as adding some "undermining bonus" for
level pawns, when the defending side can not easily avoid the exchange
by advancing her pawn.
• Case 1) White b2,c3, Black a3,b3:
Black is breaking through, b2 deserves a penalty
• Case 2) White b2,c3, Black a3,c4:
if b2xa3 then White ends up with a weak pawn on a3
and probably a weak pawn on c3 too.
In either case, White can still not safely play b2-b3 and make a
phalanx with c3, which is the essence of a backward pawn definition.
There were some promising tests a couple of months ago about adding
a lever condition for king danger in evaluate.cpp, maybe it would
be time to re-try this after all the recent changes in pawns.cpp
Remove nine boolean arguments and the corresponding skipEarlyPruning variable.
Instead, skip early pruning only when there is an excluded move, and try null
move pruning only if the previous move was not itself a null move.
protonspring [Sun, 6 May 2018 07:42:49 +0000 (09:42 +0200)]
Simplify the backward pawns code
The two lines of code in the patch seem to be just as good as master.
1. We now only look at the current square to see if it is currently backward,
whereas master looks there AND further ahead in the current file (master would
declare a pawn "backward" even though it could still safely advance a little).
This simplification allows us to avoid the use of the difficult logic with
`backmost_sq(Us, neighbours | stoppers)`.
2. The condition `relative_rank(Us,s) < RANK_5` is simplified away.
• The new code flags some pawns on the 5th rank as backward, which was not the
case in the old master. So maybe we should test a version with that included?
• Further tweaks of the backward condition with [0..5] bounds?
Tweak the connected[] array value for pawns on rank 5
A recent tuning session by Jerry Donald Watson suggested that the
value for the pawns on the fifth rank in the connected[] array were
a little bit too high in master. We lower here this value from 75 to 65.
Correct a bug introduced by Stéphane in the previous patch.
When we are using the "Bitboard + Square" overloaded operators,
the compiler uses the interpediate SquareBB[s] to transform the
square into a Bitboard, and then calculate the result.
For instance, the following code:
```
b = pos.pieces(Us, PAWN) & s
```
generates in fact the code:
```
b = pos.pieces(Us, PAWN) & SquareBB[s]`
```
The bug introduced by Stéphane in the previous patch was the
use of `b = pos.pieces(Us, PAWN) & (s + Up)` which can result
in out-of-bounds errors for the SquareBB[] array if s in the
last rank of the board.
We coorect the bug, and also add some asserts in bitboard.h to
make the code more robust for this particular bug in the future.
protonspring [Tue, 1 May 2018 21:50:23 +0000 (23:50 +0200)]
Use special rule for BlockedByKing
Simplification: remove BlockedByKing from storm array and use a special rule.
The BlockedByKing section in the storm array is substantially similar to the
Unopposed section except for two extreme values V(-290), V(-274). Turns out
removing BlockedByKing and using a special rule for these two values shows
no Elo loss. All the other values in the BlockedByKing section are apparently
irrelevant. BlockedByKing now falls under unopposed which (to me) is a bit
more logical since there is no defending pawn on this file. Also, retuning
the Unopposed section may be another improvement.
GOOD) This is a simplification because the entire BlockedByKing section of
the storm array goes away reducing a few lines of code (and less values to
tune). This also brings clarity because the special rule is self documenting.
BAD) It takes execution time to apply the special rule. This should be negli-
gible because it is based on a template parameter and is boiled down to two
bitwise AND's.
This patch may open the possibility to move the special rule to evaluate.cpp
in the evaluate::king() function, where we could refine the rule using king
danger information. For instance, with a king in H2 blocking an opponent pawn
in H3, it may be critical to know that the opponent has no safe check in G2
before giving the bonus :-)
This is a further step in the long quest for a simple way of determining
scale factors for the endgame.
Here we remove the artificial restriction in evaluate_scale_factor()
based on endgame score. Also SCALE_FACTOR_ONEPAWN can be simplified
away. The latter is a small non functional simplification with respect
to the version that was testedin the framework, verified on bench with
depth 22 for good measure.
Maybe the general case could be scaled with pawns from both colors
without losing Elo. If that is the case, then this could be merged
somehow with the scaling in evaluate_initiative(), which also uses
a additive malus down when the number of pawns in the position goes
down.
The mobilityArea is used in various places of the evaluation as a
soft proxy for "not attacked by the opponent pawns". Now that the
mobility area is getting smaller and smaller, it may be worth to
hunt for Elo gains by trying the more direct ~attackedBy[Them][PAWN]
instead of mobilityArea[Us] in these places.
Remove the distinction between the king file and the two neighbours
files in the ShelterStrength[] array. Instead we initialize the safety
variable in the evaluate_shelter() function with a -10 penalty if our
king is on a semi-open file (ie. if our king is on a file without a pawn
protection).
Also rename shelter_storm() to evaluate_shelter() while there.
See the commit history in PR#1559 for the proof that the committed
version is equivalent to the version in the tests above:
https://github.com/official-stockfish/Stockfish/pull/1559
Full credit to @protonspring for the renormalized values of the
ShelterStrength[] array used for the simplification. Thanks!
Change the operators of the Option type in uci.h to accept floating
point numbers in double precision on input as the numerical type for
the "spin" values of the UCI protocol.
The output of Stockfish after the "uci" command is unaffected.
This change is compatible with all the existing GUI (as they will
continue sending integers that we can interpret as doubles in SF),
and allows us to pass double parameters to Stockfish in the console
via the "setoption" command. This will be useful if we implement
another tuner as an alternative for SPSA.
A example of the new functionality in action in the branch `tune_float2'`:
https://github.com/snicolet/Stockfish/commit/876c322d0f20ee232da977b4d3489c4cc929765e
I have added the following lines in ucioptions.cpp:
```C++
void on_pi(const Option& o)
{
double x = Options["PI"]; // or double x = o;
std::cerr << "received value is x = " << x << std::endl;
}
By adding a direction (based on the template color), this is reduced to two
operations. This works because b is limited to enemy pawns that are ahead of
the king and on the current file.
```
shift<Down>(b) & ksq
```
I've added a line of code, but the number of executing instructions is reduced
(I think). I'm not sure if this counts as a simplification, but it should
theoretically be a little faster (barely). The code line length is also reduced
making it a little easier to read.
This patch corrects both MultiPV behaviour and "go searchmoves" behaviour
for tablebases.
We change the logic of table base probing at root positions from filtering
to ranking. The ranking code is much more straightforward than the current
filtering code (this is a simplification), and also more versatile.
If the root is a TB position, each root move is probed and assigned a TB score
and a TB rank. The TB score is the Value to be displayed to the user for that
move (unless the search finds a mate score), while the TB rank determines which
moves should appear higher in a multi-pv search. In game play, the engine will
always pick a move with the highest rank.
Ranks run from -1000 to +1000:
901 to 1000 : TB win
900 : normally a TB win, in rare cases this could be a draw
1 to 899 : cursed TB wins
0 : draw
-1 to -899 : blessed TB losses
-900 : normally a TB loss, in rare cases this could be a draw
-901 to -1000 : TB loss
Normally all winning moves get rank 1000 (to let the search pick the best
among them). The exception is if there has been a first repetition. In that
case, moves are ranked strictly by DTZ so that the engine will play a move
that lowers DTZ (and therefore cannot repeat the position a second time).
Losing moves get rank -1000 unless they have relatively high DTZ, meaning
they have some drawing chances. Those get ranks towards -901 (when they
cross -900 the draw is certain).
This patch introduces an Analysis Contempt UCI combo box to control
the behaviour of contempt during analysis. The possible values are
Both, Off, White, Black. Technically, the engine is supposed to be in
analysis mode if UCI_AnalyseMode is set by the graphical user interface
or if the user has chosen infinite analysis mode ("go infinite").
Credits: the idea for the combo box is due to Michel Van den Bergh.
The so-called "contempt" is an optimism value that the engine adds
to one color to avoid simplifications and keep tension in the position
during its search. It was introduced in Stockfish 9 and seemed to give
good results during the TCEC 11 tournament (Stockfish seemed to play a
little bit more actively than in previous seasons).
The patch does not change the play during match or blitz play, but gives
more options for correspondance players to decide for which color(s) they
would like to use contempt in analysis mode (infinite time). Here is a
description of the various options:
* Both : in analysis mode, use the contempt for both players (alternating)
* Off : in analysis mode, use the contempt for none of the players
* White : in analysis mode, White will play actively, Black will play passively
* Black : in analysis mode, Black will play actively, White will play passively
This corrects a bug in Tablebases::probe_dtz() which sometimes causes
a higher DTZ value to be returned for the position one ply before mate
than for the position two plies before mate.
The problem was reported by Kolja Kühn here:
http://talkchess.com/forum/viewtopic.php?p=757497#757497
It is explained here:
http://talkchess.com/forum/viewtopic.php?p=757506#757506
I have also adjusted some comments to make clear that probe_dtz()
returns -1 for a mate position.
Marco Costalba [Thu, 12 Apr 2018 07:22:40 +0000 (09:22 +0200)]
Further documentation and coding style on TB code
This patch adds some documentation and code cleanup to tablebase code.
It took me some time to understand the relation among the differrent
structs, although I have rewrote them fully in the past. So I wrote
some detailed documentation to avoid the same efforts for future readers.
Also noteworthy is the use a standard hash table implementation with a
more efficient 1D array instead of a 2D array. This reduces the average
lookup steps of 90% (from 343 to 38 in a bench 128 1 16 run) and reduces
also the table from 5K to 4K
entries.
I have tested on 5-men and no functional and no slowdown reported. It
should be verified on 6-men that the new hash does not overflow. It is
enough to run ./stockfish with 6-men available: if it does not assert at
startup it means everything is ok with 6-men too.
EDIT: verified for 6-men tablebase by Jörg Oster. Thanks!
We remove an unnecessary condition in the definition of safe squares
in the space evaluation. Only the squares which are occupied by our
pawns or attacked by our opponent's pawns are now excluded.
Mark Tenzer [Fri, 6 Apr 2018 23:20:48 +0000 (01:20 +0200)]
Introduce Overload
This patch applies a S(10, 5) bonus for every square that is:
- Occupied by an enemy piece which is not a pawn
- Attacked exactly once by our pieces
- Defended exactly once by enemy pieces
The idea is that these pieces must be defended. Their defenders have
dramatically limited mobility, and they are vulnerable to our future
attack.
As with connectivity, there are probably many more tests to be run in
this area. In particular:
- I believe @snicolet's queen overload tests have demonstrated a potential
need for a queen overload bonus above and beyond this one; however, the
conditions for "overload" in this patch are different (excluding pieces
we attack twice). My next test after this is (hopefully) merged will be
to intersect the Bitboard I define here with the enemy's queen attacks and
attempt to give additional bonus.
- Perhaps we should exclude pieces attacked by pawns--can pawns really be
overloaded? Should they have the same weight, or less? This didn't work
with a previous version, but it could work with this one.
- More generally, different pieces may need more or less bonus. We could
change bonuses based on what type of enemy piece is being overloaded, what
type of friendly piece is attacking, and/or what type of piece is being
defended by the overloaded piece and attacked by us, or any intersection
of these three. For example, here attacked/defended pawns are excluded,
but they're not totally worthless targets, and could be added again with
a smaller bonus.
- This list is by no means exhaustive.
This is my first time opening a PR, so I apologize if there are errors.
There are too many people to thank since I submitted my first test just
over a month ago. Thank you all for the warm welcome and here is to more
green patches!
In particular, I would like to thank:
- @crossbr, whose comment in a FishCooking thread first inspired me to
consider the overloading of pieces other than queens,
- @snicolet, whose queen overload tests inspired this one and served as
the base of my first overload attempts,
- @protonspring, whose connectivity tests inspired this one and who provided
much of the feedback needed to take this from red to green,
- @vondele, who kindly corrected me when I submitted a bad LTC test,
- @Rocky640, who has helped me over and over again in the past month.
Remove the Queen from the mobility area of minor pieces
In master, we already remove the King from the mobility area of minor pieces
because the King simply stands in the way of other pieces, and since opponent
cannot capture the King, any piece which "protects" the King cannot recapture.
Similarly, this patch introduces the idea that it is rarely a need for a Queen
to be "protected" by a minor (unless it is attacked only by a Queen, in fact).
We used to have a LoosePiece bonus, and in a similar vein the Queen was excluded
from that penalty.
Idea came when reviewing an old game of Kholmov. He was a very good midgame
player, but in the opening his misplace his Queen (and won in the end :-) :
http://www.chessgames.com/perl/chessgame?gid=1134645
Both white queen moves 10.Qd3 and 13.Qb3 are in the way of some minor piece.
I would prefer to not give a bishop mobility bonus at move 10 for the square d3,
or later a knight mobility bonus at move 13 for the square b3. And the textbook
move is 19.Qe3! which prepares 20.Nb3. This short game sample shows how much a
queen can be "in the way" of minor pieces.
Torsten Franz [Tue, 3 Apr 2018 21:33:55 +0000 (23:33 +0200)]
Simplify ThreatBySafePawn evaluation
Simplify ThreatBySafePawn evaluation by removing the 'if (weak)' speed
optimization check from threats evaluation. This is a non functional
change as it removes just a speed optimization conditional which was
probably useful before but does no longer provide benefits. This section
section had a few more lines not long ago, with ThreatByHangingPawn and
a loop through the threatened pieces, but now there is not much left.
In order to understand better the impact of various techniques used in search,
Elo estimates have been run at STC for 60000 games (statistical error ~1.8 Elo),
disabling each feature in turn. This should help future improvements and
simplifications to pick suitable targets.
Note that it will be a waste to recompute these estimates often, even a couple
of [0,5] patches are unlikely to change them by more than the error bars. The
interest of the Elo annotations in the code is not in the details, but in high-
lighting trends such as razoring (2 Elo) and singular extensions (60 Elo). These
estimates should be recomputed at most once a year.
When we reach a position with only two opposite colored bishops and
one pawn on the board, current master would give it a scale factor
of 9/64=0.14 in about one position out of 7200, and a scale factor
of 0.0 in the 7199 others. The patch gives a scale factor of 0.0 in
100% of the cases.
We also have exhaustive coverage analysis of this patch effect by
Alain Savard, comparing the perfect evaluation given by the Syzygy
tablebase with the heuristic play after this patch for the set of
all legal positions of the KBPKP endgame with opposite bishops, in
the comments thread for this pull request:
https://github.com/official-stockfish/Stockfish/pull/1520
Alain's conclusion:
> According to this definition and the data, I consider this PR is
> identical to master to "solve for draw" and slightly better than
> master to solve earlier for "wins".
Note: this patch is a side effect of an ongoing effort to improve
the evaluation of positions involving a pair of opposite bishops.
See the GitHub diff of this LTC test which almost passed at sprt[0..5]
for a discussion:
http://tests.stockfishchess.org/tests/view/5ab9030b0ebc5902932cbf93
Alain SAVARD [Thu, 29 Mar 2018 11:59:35 +0000 (07:59 -0400)]
Candidate Passed Pawn
Include some not fully supported levers in the (candidate) passed pawns
bitboard, if otherwise unblocked. Maybe levers are usually very short
lived, and some inaccuracy in the lever balance for the definition of
candidate passed pawns just triggers a deeper search.
Here is a example of a case where the patch has an effect on the definition
of candidate passers: White c5/e5 pawns, against Black d6 pawn. Let's say
we want to test if e5 is a candidate passer. The previous master looks
only at files d, e and f (which is already very good) and reject e5 as
a candidate. However, the lever d6 is challenged by 2 pawns, so it should
not fully count. Indirectly, this patch will view such case (and a few more)
to be scored as candidates.
This was inspired by this test of Jerry Donald Watson, except the case of
zero supporting pawns against two levers is excluded, and it seems that
not excluding that case is bad, while excluding is it beneficial. See the
following tests on fishtest:
> My thinking as to why this works:
>
> The evaluation is either called in an interior node or in the qsearch.
> The calls at the end of the qsearch are the more important as they
> ultimately determine the scoring of each move, whereas the internal
> values are mainly used for pruning decisions with a margin. Some strong
> engines don't even call the eval at all nodes. Now the whole point of
> the qsearch is to find quiet positions where captures do not change the
> evaluation of the position with regards to the search bounds - i.e. if
> there were good captures they would be tried.* So when a candidate lever
> appears in the evaluation at the end of the qsearch, the qsearch has
> guaranteed that it cannot just be captured, or if it can, this does not
> take the score past the search bounds. Practically this may mean that
> the side with the candidate lever has the turn, or perhaps the stopping
> lever pawn is pinned, or that side is forced for other reasons to make
> some other move (e.g. d6 can only take one of the pawns in the example
> above).
>
> Hence granting the full score for only one lever defender makes some
> sense, at least, to me.
>
> IMO this is also why huge bonuses for possible captures in the evaluation
> (e.g. threat on queen and our turn), etc. don't tend to work. Such things
> are best left to the search to figure out.
Ondrej Mosnáček [Fri, 30 Mar 2018 08:47:05 +0000 (10:47 +0200)]
Use per-thread dynamic contempt
We now use per-thread dynamic contempt. This patch has the following
effects:
* for Threads=1: **non-functional**
* for Threads>1:
* with MultiPV=1: **no regression, little to no ELO gain**
* with MultiPV>1: **clear improvement over master**
First, I tried testing at standard MultiPV=1 play with [0,5] bounds.
This yielded 2 yellow and 1 red test:
Finally, I ran some tests with fixed number of games, checking if
reverting dynamic contempt gains more elo with Skill Level=17 (i.e.
MultiPV) than applying the "prevScore" fix and this patch. These tests
showed, that this patch gains 15 ELO when playing with Skill Level=17:
**Why should this be commited?**
I believe that the gain for multi-thread MultiPV search is a sufficient
justification for this otherwise neutral change. I also believe this
implementation of dynamic contempt is more logical, although this may
be just my opinion.
**Why is per-thread contempt better at MultiPV?**
A likely explanation for the gain in MultiPV mode is that during
search each thread independently switches between rootMoves and via
the shared contempt score skews each other's evaluation.
**Why were the tests done with Skill Level=17?**
This was originally suggested by @Hanamuke and the idea is that with
Skill Level Stockfish sometimes plays also moves it thinks are slightly
sub-optimal and thus the quality of all moves offered by the MultiPV
search is checked by the test.
**Why are the ELO differences so huge?**
This is most likely because of the nature of Skill Level mode --
since it slower and weaker than normal mode, bugs in evaluation have
much greater effect.
Extends valgrind/sanitizer testing to cover syzygy code.
The script downloads 4 man syzygy as needed. The time needed for the
additional testing is small (in fact hard to see a difference compared
to the large fluctuations in testing time in travis).
Possible follow-ups:
* include more TB sensitive positions in bench.
* include the test script of recent commit "Refactor tbprobe.cpp".
* verify unchanged bench with TB (with a long run).
* make the TB part of the continuation integration tests optional.
Closes https://github.com/official-stockfish/Stockfish/pull/1518
and https://github.com/official-stockfish/Stockfish/pull/1490
Adjust criterion for applying extra reduction if not improving.
We now add an extra ply of reduction if r > 1.0, instead of the
previous condition Reductions[NonPV][imp][d][mc] >= 2.
Why does this work? Previously, reductions when not improving had
a discontinuity as the depth and/or move count increases due to the
Reductions[NonPV][imp][d][mc] >= 2 condition. Hence, values of r
such that 0.5 < r < 1.5 would be mapped to a reduction of 1, while
1.5 < r < 2.5 would be mapped to a reduction of 3. This patch allows
values of r satisfying 1.0 < r < 1.5 to be mapped to a reduction of 2,
making the reduction formula more continuous.
protonspring [Tue, 27 Mar 2018 15:29:56 +0000 (17:29 +0200)]
Simplification: remove pawn shelter/storm masks
Encode the pawn shelter/storm masks into the danger score
This highly specialized rule directly contradicts the VERY high
danger score for blocked pawns. Reducing the danger score for
blocked pawns and removing this rule is apparently an effective
compromise.
Ondrej Mosnáček [Wed, 14 Mar 2018 15:47:45 +0000 (16:47 +0100)]
Fix dynamic contempt for MultiPV
Use rootMoves[PVIdx].previousScore instead of bestValue for
dynamic contempt. This is equivalent for MultiPV=1 (bench remained the
same, even for higher depths), but more correct for MultiPV.
Note: although the ELO differences seem huge, they are inflated by the
nature of Skill Level / MultiPV search, so I don't think they can be
reasonably compared with classic ELO strength.
See https://github.com/official-stockfish/Stockfish/pull/1491 for some
verifications searches with MultiPV = 10 at depths 12 and 24 from the
starting position and the position after 1.e4, comparing the outputs
of the full PV by the old master and by this patch.
Ondrej Mosnáček [Sat, 10 Mar 2018 13:37:42 +0000 (14:37 +0100)]
Refactor tbprobe.cpp
This involves:
* replacing the union hacks with simply reusing the EntryPiece arrays
for the no-pawns case
* merging the PairsData structure with the EntryPiece/-Pawn structs
(with credit to Marco: @mcostalba)
* simplifying some HashTable functions
* thanks to previous changes, removing the ugly memsets
* simplifying the template logic for WDL/DTZ distinction
(now we distinguish based on an enum type, not the entry classes)
* removing the unneeded Atomic wrapper
-----------------------------
For reference, here is a manual way to check that patches concerning
table bases code are non-functional changes:
0) Download the Syzygy table bases (up to 6 men).
1) Make sure you have branches master and the pull request pointing to
the right commits.
2) Download the bench calculation scripts from the following URL:
and copy into src inside your Stockfish repo.
3) Make the scripts executable (chmod +x *.sh).
4) Run the following command to use TBs located at <path>:
export SYZYGY_PATH='<path>'
5) After that, run this (it will take a long time, this is a deep bench):
We check at compile time that the TimePoint type is exactly 64 bits long for
the compiler (TimePoint is our alias in Stockfish for std::chrono::milliseconds
-- it is a signed integer type of at least 45 bits according to the C++ standard,
but will most probably be implemented as a 64 bits signed integer on modern
compilers), and we use this TimePoint type consistently across the code.
Bug report by user "fischerandom" on the TCEC chat (thanks), and the
patch includes code and suggestions by user "WOnder93" and Ronald de Man.
Make kingRing always eight squares, extending the bitboard to the
F file if the king is on the H file, and to the C file if the king
is on the A file. This may deal with cases where Stockfish (like
many other engines) would shift the king around on the back rank
like g1h1, not because there is some imminent threat, but because
it makes king safety look a little better just because the king ring
had a smaller area.
This patch probably makes it easier to tune the king safety evaluation,
because the new regularity of the king ring size will make the king
safety function more continuous.
Unifies a bit further the three refuation stages in the MovePicker
class. Also treat the skipping of TT move now always via select_move(),
as discussed in pull request #1454.
projects / stockfish / log