This alllows to simplify the code because the move counter haven't to be
decremented later if a move isn't legal. As a side effect now illegal
pruned moves doesn't included anymore in move counter. So slightly less
pruning and reductions are done.
MJZ1977 [Thu, 6 Aug 2020 15:39:10 +0000 (17:39 +0200)]
NNUE evaluation threshold
The idea is to use NNUE only on quite balanced material positions. This bring a big speedup on research since NNUE eval is slower than classical eval for most of the hardwares and specially on unbalanced positions with LazyEval.
nodchip [Wed, 5 Aug 2020 15:11:15 +0000 (17:11 +0200)]
Add NNUE evaluation
This patch ports the efficiently updatable neural network (NNUE) evaluation to Stockfish.
Both the NNUE and the classical evaluations are available, and can be used to
assign a value to a position that is later used in alpha-beta (PVS) search to find the
best move. The classical evaluation computes this value as a function of various chess
concepts, handcrafted by experts, tested and tuned using fishtest. The NNUE evaluation
computes this value with a neural network based on basic inputs. The network is optimized
and trained on the evalutions of millions of positions at moderate search depth.
The NNUE evaluation was first introduced in shogi, and ported to Stockfish afterward.
It can be evaluated efficiently on CPUs, and exploits the fact that only parts
of the neural network need to be updated after a typical chess move.
[The nodchip repository](https://github.com/nodchip/Stockfish) provides additional
tools to train and develop the NNUE networks.
This patch is the result of contributions of various authors, from various communities,
including: nodchip, ynasu87, yaneurao (initial port and NNUE authors), domschl, FireFather,
rqs, xXH4CKST3RXx, tttak, zz4032, joergoster, mstembera, nguyenpham, erbsenzaehler,
dorzechowski, and vondele.
This new evaluation needed various changes to fishtest and the corresponding infrastructure,
for which tomtor, ppigazzini, noobpwnftw, daylen, and vondele are gratefully acknowledged.
The first networks have been provided by gekkehenker and sergiovieri, with the latter
net (nn-97f742aaefcd.nnue) being the current default.
The evaluation function can be selected at run time with the `Use NNUE` (true/false) UCI option,
provided the `EvalFile` option points the the network file (depending on the GUI, with full path).
The performance of the NNUE evaluation relative to the classical evaluation depends somewhat on
the hardware, and is expected to improve quickly, but is currently on > 80 Elo on fishtest:
The needed networks can be found at https://tests.stockfishchess.org/nns
It is recommended to use the default one as indicated by the `EvalFile` UCI option.
Guidelines for testing new nets can be found at
https://github.com/glinscott/fishtest/wiki/Creating-my-first-test#nnue-net-tests
Integration has been discussed in various issues:
https://github.com/official-stockfish/Stockfish/issues/2823
https://github.com/official-stockfish/Stockfish/issues/2728
The integration branch will be closed after the merge:
https://github.com/official-stockfish/Stockfish/pull/2825
https://github.com/official-stockfish/Stockfish/tree/nnue-player-wip
We double the bonus for potential threats by minors and rooks against
our queen, in case of "queen vs pieces imbalance". Hopefully this will
improve a little bit the evaluation for this well-known Stockfish weakness.
This is a functional simplification that removes the conditional_more_than_two()
function, which was quite strange and kooky. Note the very minor change to the bench
value.
See this thread for relevant comments on the passing branch:
protonspring/Stockfish@d89730d...ff35b50
This corrects a functional change in
https://github.com/official-stockfish/Stockfish/commit/ddcbacd04d1c860e808202ce8c1206c8acdca627
changing evaluation of KPPvK. Bench remains unchanged at low depth
With this patch, 8/8/5k1p/8/7p/7K/8/8 b - - 1 11 is again correctly evaluated as a draw.
In some French games, Stockfish likes to bring the Knight to a bad outpost spot. This is evident in TCEC S18 Superfinal Game 63, where there is a Knight outpost on the queenside but is actually useless. Stockfish is effectively playing a piece down while holding ground against Leela's break on the kingside.
This patch turns the +56 mg bonus for a Knight outpost into a -7 mg penalty if it satisfies the following conditions:
* The outpost square is not on the CenterFiles (i.e. not on files C,D,E and F)
* The knight is not attacking non pawn enemies.
* The side where the outpost is located contains only few enemies, with a particular conditional_more_than_two() implementation
Thank you to apospa...@gmail.com for bringing this to our attention and for providing insights.
See https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/dEXNzSIBgZU
Reference game: https://tcec-chess.com/#div=sf&game=63&season=18
various related tests show strong test results, but so far no generalizations or simplifications of conditional_more_than_two() are found. See PR for details.
Probcut is a heuristic that wasn't changed a lot in past years,
all attempts to change it using information / writing info to transposition table failed.
This patch has a number of differences that can be summarized as follows:
* For TT write/read we use depth - 3. Because probcut search is depth - 4 but we actually do the move prior to it so effectively we do depth - 3 search;
* In any case of depth of eval from transposition table being >= depth - 3 we either produce cutoff or refuse to even do probcut search, this is allowing us to write info of probcut to transposition table because we know that we wouldn't be overwriting some deeper data with our depth - 3 search - this is an important aspect of this patch;
* For some not really known reason this patch completely ignores tte->bound() - which was the case for previous patch that made probcut interact with TT, maybe 2) is the reason, although it's unproven.
However, an additional bugfix was needed to avoid checking a condition on ttMove if was not available. This passed non-regression bounds on top of the first version:
the goal of this PR is to better document how we process
the ep square (if any) given position fen command, and to
output more meaningful (and consistent) debug fen on the "d"
command. The implementation follows
https://en.wikipedia.org/wiki/X-FEN#Encoding_en-passant
following x-fen, it is "valid" to record ep even if ep would put king en prise.
Fix for overevaluated blocked pawns on the 5th and 6th rank.
This is a rewrite of the original idea that uses only two parameters.
Thanks to rocky640 for pointing this out.
UCI_ShowWDL might not be shown by GUIs that don't know the option,
but crash on the WDL output, effectively making it hard for users to
turn it off and run the engine. This sets it by default to false.
A number of engines, GUIs and tournaments start to report WDL estimates
along or instead of scores. This patch enables reporting of those stats
in a more or less standard way (http://www.talkchess.com/forum3/viewtopic.php?t=72140)
The model this reporting uses is based on data derived from a few million fishtest LTC games,
given a score and a game ply, a win rate is provided that matches rather closely,
especially in the intermediate range [0.05, 0.95] that data. Some data is shown at
https://github.com/glinscott/fishtest/wiki/UsefulData#win-loss-draw-statistics-of-ltc-games-on-fishtest
Making the conversion game ply dependent is important for a good fit, and is in line
with experience that a +1 score in the early midgame is more likely a win than in the late endgame.
Even when enabled, the printing of the info causes no significant overhead.
We lower the endgame value of the evaluation when we detect that there
is only one queen left on the board (more precisely, we use a scale
factor of 37/64, or about 0.58, for the endgame part of the evaluation).
Hopefully this helps a little bit for the assessment of positions with
queen imbalance, which are one of the well-known Stockfish weaknesses.
This patch increases the endgame value of pawns on the fifth rank.
The increase is very small (+1 evaluation point, about 0.005 pawn)
for the pawns on external columns (a-b-c-f-g-h) and a bit bigger
(+7 evaluation points, about 0.033 pawn) for the pawns on d5/e5.
mstembera [Fri, 26 Jun 2020 05:08:17 +0000 (22:08 -0700)]
Remove old zobrist trick for castling rights
Removes an 8 year old micro optimization aimed at 32-bit architectures
because back then doing an xor of a Key could not be done in one instruction.
See original commit here 821e1c7
mstembera [Sat, 27 Jun 2020 00:26:46 +0000 (17:26 -0700)]
Fix fragile code to use proper random 64 bit keys.
This fixes an old issue where we want to make a position unique but only
change a small number of bits in the key instead of all 64 of them randomly.
This is fragile and can lead to non uniqueness issues in the TT.
Key make_key(uint64_t seed) takes any integer and produces a unique random 64 bit key.
It is computationally efficient and is based on a congruential pseudo random number
generator using well tested constants by Donald Knuth
(see https://en.wikipedia.org/wiki/Linear_congruential_generator)
the option was, since at least 2014, not correctly implemented,
ignoring all dynamic adjustments to optimum time in search.
Instead of fixing it, remove it, no need to expose an option that
will influence time management negatively.
Vizvezdenec [Tue, 23 Jun 2020 11:55:52 +0000 (14:55 +0300)]
Do less futility pruning for captures.
The idea of this patch is that if capture can be described as
"less valuable piece takes more valuable piece" it's not really correct
to add only piece value of captured piece to static evaluation
since there can be more threats in other places and opponent can't really
do much but recapture our capturing piece which leaves us space for
more captures thus winning more material and increasing static eval.
FauziAkram [Mon, 22 Jun 2020 09:52:31 +0000 (12:52 +0300)]
Introduce bonus for queen infiltration
Idea is that queen feels much better when it can't be kicked away now
or later by pawn moves, especially in endgame. Special thanks to
Linmiao Xu for the original idea of this patch.
Vizvezdenec [Wed, 17 Jun 2020 02:36:30 +0000 (05:36 +0300)]
More reduction for evading pawn moves.
pawn moves are irreversable unlike other evading moves; pawn is the least valuable piece in the game.
So it makes a lot of sence to assume that evading pawn moves are on average not as good as other evading moves thus can be reduced more.
We increase a little bit the midgame value of pawns on a4, h4, a6 and h6.
Original idea by Malcolm Campbell, who tried the version restricted to the
pawns on the H column a couple of weeks ago and got a patch which almost
passed LTC. The current pull request just adds the same idea for pawns on
the A column.
Possible follow-ups: maybe tweak the a5/h5 pawn values, and/or add a malus
for very low king mobility in midgame?
Scheme back to being derived from https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/
Also the default optimized version of the index calculation now uses fewer instructions.
https://godbolt.org/z/Tktxbv
Might benefit from mulx (requires -mbmi2)
xoto10 [Sun, 14 Jun 2020 20:50:27 +0000 (21:50 +0100)]
Retuned values after eval quantize patch.
The last search tune patch was tested before the implementation of #2733 which
presumably changed the search characteristics noticeably. Another tuning run was
done, see https://tests.stockfishchess.org/tests/view/5ee5b434ca6c451633a9a08c
and the updated values passed these tests:
The changes were rebased on top of a successful patch by Viz (see #2734)
and two different ways of doing this were tested. The successful test
modified the constants in the patch by Viz in a similar manner to the
tuning run:
Further work: the recent patch to quantize eval #2733 affects search quit
quite a bit, so doing another tune in, say, three months time might be a
good idea.
Effectively reintroduces evaluation grain, with a slightly different context.
https://github.com/official-stockfish/Stockfish/commit/45dbd9cd0303d0db469670af8ec3598731a4eace
Vizvezdenec [Sat, 13 Jun 2020 02:03:59 +0000 (05:03 +0300)]
Adjust history threshold for quiet moves futility pruning
This patch adjusts the threshold for futility pruning of quiet moves
using the continuation history array contHist[5], in the same way as
it is used in movepicker.
We replace the current decrease of the complexity term in initiative
when shuffling by a direct damping of the evaluation. This scheme may
have two benefits over the initiative approach:
a) the damping effect is more brutal for fortresses with heavy pieces
on the board, because the initiative term is almost an endgame term;
b) the initiative implementation had a funny side effect, almost a bug,
in the rare positions where mg > 0, eg < 0 and the tampered eval
returned a positive value (ie with heavy pieces still on the board):
sending eg to zero via shuffling would **increase** the tampered
eval instead of decreasing it, which is somewhat illogical. This
patch avoids this phenomenon.
Sami Kiminki [Fri, 5 Jun 2020 17:17:00 +0000 (20:17 +0300)]
Increase the maximum hash size by a factor of 256
Conceptually group hash clusters into super clusters of 256 clusters.
This scheme allows us to use hash sizes up to 32 TB
(= 2^32 super clusters = 2^40 clusters).
Use 48 bits of the Zobrist key to choose the cluster index. We use 8
extra bits to mitigate the quantization error for very large hashes when
scaling the hash key to cluster index.
The hash index computation is organized to be compatible with the existing
scheme for power-of-two hash sizes up to 128 GB.
protonspring [Sun, 7 Jun 2020 03:25:32 +0000 (21:25 -0600)]
Wrap all access to LineBB and add assert
This is a non-functional code style change which provides a safe access handler for LineBB.
Also includes an assert in debug mode to verify square correctness.
xoto10 [Wed, 3 Jun 2020 10:05:58 +0000 (11:05 +0100)]
join scale_factor, initiative and mg+eg reduction
Merging this code into one function `winnable()`.
Should allow common concepts used to adjust the eg value,
either by addition or scaling, to be combined more effectively.
Moez Jellouli [Sun, 31 May 2020 14:51:38 +0000 (16:51 +0200)]
Minimal thinking time, even if only one rootMove.
without search, the eval returned can be misleading (e.g. mate instead of draw),
leading to wrong adjudication. With a minimal search, this is avoided.
This patch leads to 1ms long searches if there is only 1 move,
similar patches all indicate a small Elo gain.
Give bonus for bishops that are alligned with enemy kingring.
Inspired by the succesful patch "Give bonus for rooks that are alligned with enemy kingring" from Vizvezdenec,
this idea has been reused for bishops. Here, we only consider attacks that are not blocked by any pawn.
Also we have a 50% higher bonus than for the rooks.
ElbertoOne [Sun, 31 May 2020 14:39:03 +0000 (16:39 +0200)]
Isolated pawns tweak
Give opposed doubled isolated pawns only the Doubled penalty.
The other isolated pawns get the Isolated penalty and the WeakUnopposed penalty.
The popcount condition has been replaced with an opposed check,
which is non-functional, but probably gives a speed-up.
protonspring [Thu, 28 May 2020 15:48:31 +0000 (09:48 -0600)]
Consolidate all attacks bitboards
This is a non-functional simplification that simplifies getting attacks bitboards.
* consolidates all attacks to attacks_bb (remove Position::attacks_from(..)).
* attacks_bb<PieceType>(square) gets pseudo attacks
* attacks_bb<PieceType>(square, bitboard) gets attacks considering occupied squares in the bitboard).
* pawn_attacks_bb(Color, Square) gets pawn attacks like other pawn attack bitboards.
* Wraps all access to PawnAttacks arrays and PseudoAttacks arrays and adds asserts as appropriate.
Vizvezdenec [Mon, 25 May 2020 18:14:07 +0000 (21:14 +0300)]
Give bonus for rooks that are alligned with enemy kingring
The idea of this patch is that if rooks are not directly attacking the opponent king,
they can support king attacks staying behind pawns or minor pieces and be really
deadly if position slightly opens up at enemy king ring ranks. Loosely based on
some stockfish games where it underestimated attacks on it king when enemy has one
or two rooks supporting pawn pushes towards it king.
FauziAkram [Sat, 23 May 2020 09:22:34 +0000 (12:22 +0300)]
Queen Mobility Tweak
It's ok to have low mobility values for the Queen in the middlegame, but it's absolutely not ok to have low mobility values for the Queen in the endgame.
Decrease penalty for bad mobility in MG and increase it in EG.
This patch gives an additional penalty if a doubled isolated pawn is stopped
only by a single opponent pawn on the same file. Thanks to NKONSTANTAKIS,
who shared this idea on the forum!
This patch is a combinaison of two recent parameters tweaks which had
failed narrowly (yellow) at long time control:
• improvement in move ordering during search by softening the distinction
between bad captures and good captures during move generation, leading
to improved awareness of Stockfish of potential piece sacrifices (idea
by Rahul Dsilva)
• increase in the weight of pawns in the "initiative" part of the evaluation
function. With this change Stockfish should have more incentive to exchange
pawns when losing, and to keep pawns when winning.
Sami Kiminki [Tue, 19 May 2020 09:08:01 +0000 (12:08 +0300)]
Avoid sending info strings before 'uci' has been received
Do not send the following info string on the first call to
aligned_ttmem_alloc() on Windows:
info string Hash table allocation: Windows large pages [not] used.
The first call occurs before the 'uci' command has been received. This
confuses some GUIs, which expect the first engine-sent command to be
'id' as the response to the 'uci' command. (see https://github.com/official-stockfish/Stockfish/issues/2681)
xoto10 [Sun, 17 May 2020 19:46:25 +0000 (20:46 +0100)]
Increase base time use and limit max used.
This change increases the base part of optimumTime at all depths. It also reduces the size of max_scale and thus maximumTime by using a linear scale instead of pow(x, 0.3) and by limiting max_scale to no more than 7 (previously as high as 8 or 9 at very high depths).
protonspring [Fri, 15 May 2020 23:23:49 +0000 (17:23 -0600)]
Don't adjust MoveOverhead by increment
This is a change to address a potential timing issue for slow networks.
Move Overhead was limited by TC increment,
which might be problematic if small increments (or sudden death)
on slow networks (needing high Move Overhead) are used.
protonspring [Wed, 13 May 2020 17:52:41 +0000 (11:52 -0600)]
Simplify Time Management
This is a functional simplification of the time management system.
With this patch, there is a simple equation for each of two distinct
time controls: basetime + increment, and x moves in y seconds (+increment).
These equations are easy to plot and understand making future modifications
or adding additional time controls much easier.
SlowMover is reset to 100 so that is has no effect unless a user changes it.
There are two scaling variables:
* Opt_scale is a scale factor (or percentage) of time to use for this current move.
* Max_scale is a scale factor to apply to the resulting optimumTime.
There seems to be some elo gain in most scenarios.
Better performance is attributable to one of two things:
* minThinkingTime was not allowing reasonable time calculations for very short games like 10+0 or 10+0.01. This is because adding almost no increment and substracting move overhead for 50 moves quickly results in almost 0 time very early in the game. Master depended on minThinkingTime to handle these short games instead of good time management. This patch addresses this issue by lowering minThinkingTime to 0 and adjusting moverOverhead if there are very low increments.
* Notice that the time distribution curves tail downward for the first 10 moves or so. This causes less time to attribute for very early moves leaving more time available for middle moves where more important decisions happen.
Here is a summary of tests for this version at different time controls:
Sami Kiminki [Thu, 14 May 2020 09:00:35 +0000 (12:00 +0300)]
Fix a Windows-only crash on exit without 'quit'
There was a bug in commit d4763424d2728fe2dfd0a6fe747666feb6a2fdbb
(Add support for Windows large pages) that could result in trying to
free memory allocated with VirtualAlloc incorrectly with free().
Fix this by reverting the TT.resize(0) logic in the previous commit,
and instead, just call aligned_ttmem_free() in
TranspositionTable::~TranspositionTable().
simplify the usage of the 50 moves counter,
moving it frome the scale factor to initiative.
This patch was inspired by recent games where a blocked or semi-blocked position
was 'blundered', by moving a pawn, into a lost endgame. This patch improves this situation,
finding a more robust move more often.
for example (1s searches with many threads):
```
FEN 8/p3kp2/Pp2p3/1n2PpP1/5P2/1Kp5/8/R7 b - - 68 143
Sami Kiminki [Mon, 4 May 2020 17:49:27 +0000 (20:49 +0300)]
Add support for Windows large pages
for users that set the needed privilige "Lock Pages in Memory"
large pages will be automatically enabled (see Readme.md).
This expert setting might improve speed, 5% - 30%, depending
on the hardware, the number of threads and hash size. More for
large hashes, large number of threads and NUMA. If the operating
system can not allocate large pages (easier after a reboot), default
allocation is used automatically. The engine log provides details.
The problem was caused by .depend being created with a rule for tbprobe.o not for syzygy/tbprobe.o.
This patch keeps an explicit list of sources (SRCS), generates OBJS,
and compiles all object files to the src/ directory, consistent with .depend.
VPATH is used to search the syzygy directory as needed.
Marco Costalba [Sat, 4 Apr 2015 06:54:15 +0000 (08:54 +0200)]
Fishtest Tuning Framework
The purpose of the code is to allow developers to easily and flexibly
setup SF for a tuning session. Mainly you have just to remove 'const'
qualifiers from the variables you want to tune and flag them for
tuning, so if you have:
At startup all the parameters are printed in a
format suitable to be copy-pasted in fishtest.
In case the post update function is slow and you have many
parameters to tune, you can add:
UPDATE_ON_LAST();
And the values update, including post update function call, will
be done only once, after the engine receives the last UCI option.
The last option is the one defined and created as the last one, so
this assumes that the GUI sends the options in the same order in
which have been defined.
A combination of terms related to king safety one tuned safe check weights,
the other tuned knight and bishop king protector weights separately with
some compensation in the high outpost bonuses given to the minor pieces.
The idea is somewhat similar to futility pruning for quiet moves -
if a late enough capture doesn't give check and the static eval is
much lower than alpha we can almost safely assume that this capture
wouldn't be a good move.
This patch increases number of nodes where we produce multicut cutoffs.
The idea is that if our ttMove failed to produce a singular extension
but ttValue is greater than beta we can afford to do one more reduced search
near beta excluding ttMove to see if it will produce a fail high -
and if it does so produce muticut by analogy to existing logic.
This is a functional simplification which fixes an awkward numerical cliff.
With master king_safety, no pawns is scored higher than pawn(s) that is/are far from the king. This may motivate SF to throw away pawns to increase king safety. With this patch, there is a consistent value for minPawnDistance where losing a pawn never increases king safety.
projects / stockfish / log