Sami Kiminki [Thu, 14 May 2020 09:00:35 +0000 (12:00 +0300)]
Fix a Windows-only crash on exit without 'quit'
There was a bug in commit d4763424d2728fe2dfd0a6fe747666feb6a2fdbb
(Add support for Windows large pages) that could result in trying to
free memory allocated with VirtualAlloc incorrectly with free().
Fix this by reverting the TT.resize(0) logic in the previous commit,
and instead, just call aligned_ttmem_free() in
TranspositionTable::~TranspositionTable().
simplify the usage of the 50 moves counter,
moving it frome the scale factor to initiative.
This patch was inspired by recent games where a blocked or semi-blocked position
was 'blundered', by moving a pawn, into a lost endgame. This patch improves this situation,
finding a more robust move more often.
for example (1s searches with many threads):
```
FEN 8/p3kp2/Pp2p3/1n2PpP1/5P2/1Kp5/8/R7 b - - 68 143
Sami Kiminki [Mon, 4 May 2020 17:49:27 +0000 (20:49 +0300)]
Add support for Windows large pages
for users that set the needed privilige "Lock Pages in Memory"
large pages will be automatically enabled (see Readme.md).
This expert setting might improve speed, 5% - 30%, depending
on the hardware, the number of threads and hash size. More for
large hashes, large number of threads and NUMA. If the operating
system can not allocate large pages (easier after a reboot), default
allocation is used automatically. The engine log provides details.
The problem was caused by .depend being created with a rule for tbprobe.o not for syzygy/tbprobe.o.
This patch keeps an explicit list of sources (SRCS), generates OBJS,
and compiles all object files to the src/ directory, consistent with .depend.
VPATH is used to search the syzygy directory as needed.
Marco Costalba [Sat, 4 Apr 2015 06:54:15 +0000 (08:54 +0200)]
Fishtest Tuning Framework
The purpose of the code is to allow developers to easily and flexibly
setup SF for a tuning session. Mainly you have just to remove 'const'
qualifiers from the variables you want to tune and flag them for
tuning, so if you have:
At startup all the parameters are printed in a
format suitable to be copy-pasted in fishtest.
In case the post update function is slow and you have many
parameters to tune, you can add:
UPDATE_ON_LAST();
And the values update, including post update function call, will
be done only once, after the engine receives the last UCI option.
The last option is the one defined and created as the last one, so
this assumes that the GUI sends the options in the same order in
which have been defined.
A combination of terms related to king safety one tuned safe check weights,
the other tuned knight and bishop king protector weights separately with
some compensation in the high outpost bonuses given to the minor pieces.
The idea is somewhat similar to futility pruning for quiet moves -
if a late enough capture doesn't give check and the static eval is
much lower than alpha we can almost safely assume that this capture
wouldn't be a good move.
This patch increases number of nodes where we produce multicut cutoffs.
The idea is that if our ttMove failed to produce a singular extension
but ttValue is greater than beta we can afford to do one more reduced search
near beta excluding ttMove to see if it will produce a fail high -
and if it does so produce muticut by analogy to existing logic.
This is a functional simplification which fixes an awkward numerical cliff.
With master king_safety, no pawns is scored higher than pawn(s) that is/are far from the king. This may motivate SF to throw away pawns to increase king safety. With this patch, there is a consistent value for minPawnDistance where losing a pawn never increases king safety.
Don't attempt probcut if ttMove is not good enough.
This idea is loosely based on xoroshiro idea about raisedBeta and ttmoves.
If our ttmove have low enough ttvalue and is deep enough (deeper than our probcut depth) it makes little sense to try probcut moves, since the ttMove already more or less failed to produce one according to transposition table.
This patch refines the recently introduced interaction between
the space bonus and the number of blocked pawns in a position.
* pawns count as blocked also if their push square is attacked by 2 enemy pawns;
* overall dependence is stronger as well as offset;
* bonus increase is capped at 9 blocked pawns in position;
Scale up space weight with number of blocked pawns
This idea is loosely based on stockfish losses in closed positions in different tournaments. Space weight symmetrically increases for both sides the more blocked position is.
+-------+
| o . . | o their pawns
| x . . | x our pawns
| . x . | <- Can sacrifice to create passer?
+-------+
yes
1 2 3 4 5
+-------+ +-------+ +-------+ +-------+ +-------+
| o . . | | o r . | | o r . | | o . b | | o . b | lowercase: theirs
| x b . | | x . . | | x . R | | x . R | | x . . | uppercase: ours
| . x . | | . x . | | . x . | | . x . | | . x B |
+-------+ +-------+ +-------+ +-------+ +-------+
no no yes no yes
The value of our top pawn depends on our ability to advance our bottom
pawn, levering their blocker. Previously, this pawn configuration was
always scored as passer (although a blocked one).
Add requirements for the square s above our (possibly) sacrificed pawn:
- s must not be occupied by them (1).
- If they attack s (2), we must attack s (3).
- If they attack s with a minor (4), we must attack s with a minor (5).
The attack from their blocker is ignored because it is inherent in the
structure; we are ok with sacrificing our bottom pawn.
if these are ttMoves and played in positions with a high value of the rule50 counter. The unusual extension of 2 is safe in this context as awarding it will reset the rule50 counter, making sure it is awarded very rarely in a search path.
This patch partially addresses https://github.com/official-stockfish/Stockfish/issues/2620 as it should make it less likely to play a move that resets the counter, but that is worse than alternative moves after a slightly deeper search.
In master, if the received ttMove meets the prescribed conditions in the various MovePicker constructors, it is returned as the first move, otherwise we set it to MOVE_NONE. If set to MOVE_NONE, we no longer track what the ttMove was, and it will might be returned later in a list of generated moves. This may be a waste. With this patch, if the ttMove fails to meet the prescribed conditions, we simply skip the TT stages, but still store the move and make sure it's never returned.
This patch introduces a heuristic that is similar to countermove based pruning but for captures - capture history pruning. The idea is that we can (almost) safely prune really late captures with negative history if they don't give check so will most likely not produce some king-attacking tactic.
Before this commit, some pawns were considered "candidate" passed pawns and given half bonus. After this commit, all of these pawns are scored as passed pawns, and they do not receive less bonus.
This PR and commit are dedicated to our colleague Stefan Geschwentner (@locutus2), one of the most respected and accomplished members of the Stockfish developer community. Stockfish is a volunteer project and has always thrived because of Stefan's talent, insight, generosity, and dedication. Welcome back, Stefan!
protonspring [Tue, 31 Mar 2020 21:08:55 +0000 (15:08 -0600)]
remove KNPK endgame code
In more than 100k local KNPK games, there is no discernible difference between master and master with this endgame removed: master:42971, patch:42973, draws: 3969. Removal does not seem to regress in normal games.
The idea behind this patch is that if static eval is really bad so capturing of current piece on spot will still produce a position with an eval much lower than alpha then our best chance is to create some kind of king attack. So captures without check are mostly worse than captures with check and can be reduced more.
Vizvezdenec [Sun, 29 Mar 2020 17:04:20 +0000 (20:04 +0300)]
Count only the most advanced passed pawn for each file.
This patch adjusts definition of passed pawns - if there is a pawn of our color in the same file in front of a current pawn it's no longer counts as passed.
Vizvezdenec [Fri, 20 Mar 2020 09:12:56 +0000 (12:12 +0300)]
Adjust singular extension search depth
This patch applies a different singular extension search logic in case the position is ttPv && !PvNode.
It changes the depth of this search, higher for this types of nodes, and lower for other nodes.
Vizvezdenec [Tue, 17 Mar 2020 16:38:21 +0000 (19:38 +0300)]
Adjust singular LMR for positions seen in PV
This patch continues work on altering search for ttPv nodes, using recent idea to alter it more in not PvNodes. Previous tweak based on this idea adjusted singularBeta - this one adjusts value of singularLMR, so they are both related to singular extension search.
Reduce the "bad bishop" penalty when the bishop is protected by
one of our pawns, as it may indicate that the bishop has found
a safe spot outside the pawn chain.
protonspring [Mon, 9 Mar 2020 21:11:08 +0000 (22:11 +0100)]
Equations for edges and corners.
This is a functional simplification that removes the large arrays in endgames.cpp.
It also fixes a recently introduced bug (960d59d54143d84aab26deae65279a611fc989f4) in KNBvK,
now using flip_file() instead of ~.
One fen added to bench to increase endgame coverage.
Gary Heckman [Thu, 5 Mar 2020 17:37:08 +0000 (12:37 -0500)]
Fix ambiguity between clamp implementations
There is an ambiguity between global and std clamp implementations when compiling in c++17,
and on certain toolchains that are not strictly conforming to c++11.
This is solved by putting our clamp implementation in a namespace.
protonspring [Thu, 5 Mar 2020 19:07:48 +0000 (12:07 -0700)]
Cleanup KBPsK endgame
* Clarify distinction between strong side pawns and all pawns.
* Simplify and speed-up determination of pawns on the same file.
* Clarify comments.
* more_than_one() is probably faster than pos.count.
protonspring [Tue, 3 Mar 2020 00:32:02 +0000 (17:32 -0700)]
Use equations for PushAway and PushClose
A functional simplification replacing the corresponding arrays. Tested in two variants,
also the simpler one performs well, even though differences to master should be minimal.
AndyGrant [Mon, 24 Feb 2020 22:32:17 +0000 (23:32 +0100)]
Fix TT write in MultiPV case.
fixes an error reported earlier as https://github.com/official-stockfish/Stockfish/issues/2404 by @AndyGrant.
MultiPV at root shouldn't write to the TT for later lines, as that is neither the eval nor the bestmove for that position.
Fixing this error doesn't matter for playing games (http://tests.stockfishchess.org/tests/view/5dcdbd810ebc590256324a11).
However, it can lead to wrong mate announcements as reported by @uriblass. In particular the following testcase gives
wrong results for the second search, prior to this patch:
```
setoption name MultiPV value 2
position fen 5R2/2kB2p1/p2bR3/8/3p1B2/8/PPP5/2K5 b - - 0 49
go depth 40
position fen 2B2R2/3r2p1/p1kbR3/8/3p1B2/8/PPP5/2K5 b - - 8 48
go depth 40
```
31m059 [Sun, 23 Feb 2020 02:27:32 +0000 (21:27 -0500)]
Use single param for Outpost and ReachableOutpost.
In November 2019, as a result of the simplification of rank-based outposts by 37698b0,
separate bonuses were introduced for outposts that are currently occupied and outposts
that are reachable on the next move. However, the values of these two bonuses are
quite similar, and they have remained that way for three months of development.
It appears that we can safely retire the separate ReachableOutpost parameter and
use the same Outpost bonus in both cases, restoring the basic principles of Stockfish
outpost evaluation to their pre-November state, while also reducing
the size of the parameter space.
Günther Demetz [Fri, 21 Feb 2020 13:01:59 +0000 (14:01 +0100)]
Improve move order near the root
Current move histories are known to work well near the leaves, whilst at
higher depths they aren't very helpful. To address this problem this
patch introduces a table dedicated for what's happening at plies 0-3.
It's structured like mainHistory with ply index instead of color.
It get cleared with each new search and is filled during iterative
deepening at higher depths when recording successful quiet moves near
the root or traversing nodes which were in the principal variation
(ttPv).
Theory: KNNK is a dead draw, however the presence of the additional weakSide pawn opens up some mate opportunities. The idea is to block the pawn (preferably behind the Troitsky line) with one of the knights and press the weakSide king into a corner. If we can stalemate the king, we release the pawn with the knight (to avoid actual stalemate), and use the knight to complete the mate before the pawn promotes. This is also why there is an additional penalty for advancement of the pawn.
the code that prevents false mate announcements depending on the TT
state (GHI), incorrectly used VALUE_MATE_IN_MAX_PLY. The latter
constant, however, also includes, counterintuitively, the TB win range.
This patch fixes that, by restoring the behavior for TB win scores,
while retaining the false mate correctness, and improving the mate
finding ability. In particular
no alse mates are announced with the poisened hash testcase
```
position fen 8/8/8/3k4/8/8/6K1/7R w - - 0 1
go depth 40
position fen 8/8/8/3k4/8/8/6K1/7R w - - 76 1
go depth 20
ucinewgame
```
mates are found with the testcases reported in #2543
```
position fen 4k3/3pp3/8/8/8/8/2PPP3/4K3 w - - 0 1
setoption name Hash value 1024
go depth 55
ucinewgame
```
and
```
position fen 4k3/4p3/8/8/8/8/3PP3/4K3 w - - 0 1
setoption name Hash value 1024
go depth 45
ucinewgame
```
furthermore, on the mate finding benchmark (ChestUCI_23102018.epd),
performance improves over master, roughly reaching performance with the
false mate protection reverted
```
Analyzing 6566 mate positions for best and found mates:
On a testcase with TB enabled, progress is made consistently, contrary
to master
```
setoption name SyzygyPath value ../../../syzygy/3-4-5/
setoption name Hash value 2048
position fen 1R6/3k4/8/K2p4/4n3/2P5/8/8 w - - 0 1
go depth 58
ucinewgame
```
The incorrectly named VALUE_MATE_IN_MAX_PLY and VALUE_MATED_IN_MAX_PLY
were renamed into VALUE_TB_WIN_IN_MAX_PLY and VALUE_TB_LOSS_IN_MAX_PLY,
and correclty defined VALUE_MATE_IN_MAX_PLY and VALUE_MATED_IN_MAX_PLY
were introduced.
One further (corner case) mistake using these constants was fixed (go
mate X), which could lead to a premature return if X > MAX_PLY / 2,
but TB were present.
Thanks to @svivanov72 for one of the reports and help fixing the issue.
Vizvezdenec [Fri, 7 Feb 2020 17:04:43 +0000 (20:04 +0300)]
Modify singular beta for ttPv positions.
This patch lowers singular beta for positions that have been in pv and are not pv nodes.
The idea of using ttpv && !PvNode improved scaling with TC and could be
useful for other search heuristics.
can trigger an abort when compiling with debug=yes, and using 7men TB.
The assert should check that less than 8 pieces are in the key for each
side, matching the assumption that underlies the FEN string construction.
Also take explicitly care of a 'v' character in material strings.
Fixes an issue reported in the forum:
https://groups.google.com/d/msg/fishcooking/yoVC7etIpz0/7mS7ntZMBAAJ
joergoster [Mon, 27 Jan 2020 17:53:25 +0000 (18:53 +0100)]
Simplify hashfull calculation.
We can simplify the calculation of the hashfull info by looping over exact 1,000 entries,
and then divide the result by ClusterSize. Somewhat memory accesses, somewhat more accurate.
Guenther Demetz [Tue, 28 Jan 2020 12:38:03 +0000 (13:38 +0100)]
More bonus for bestMoves on past PV nodes
It looks like it is important to keep past PV (ttPv) nodes as close as possible to current PV nodes.
Credits to Mark Tenzer (31m059) & Stefan Geschwentner who first tried ideas on ttPv nodes.
31m059 [Tue, 28 Jan 2020 01:48:01 +0000 (20:48 -0500)]
Less NMP if the position was previously in PV.
The intention of the patch is to avoid aggressive null move pruning (NMP)
in positions that have previously been found to be important (PV nodes).
If we already do not apply NMP for current PV nodes, it makes sense to apply
it less often for positions that have previously been PV nodes too.
On my box, huge pages have a significant perf impact when using a big
hash size. They also speed up TT initialization big time:
vanilla (s) huge pages (s) speed-up
=======================================================================
time stockfish bench 16384 1 1 | 5.37 1.48 3.6x
In practice, huge pages with auto-defrag may always be enabled in the
system, in which case this patch has no effect. This
depends on the values in /sys/kernel/mm/transparent_hugepage/enabled
and /sys/kernel/mm/transparent_hugepage/defrag.
Use a SEE pruned capture move for history updates: this patch collects
pruned capture moves also in the failed captures list, so that they get
an update in capture history.
Vizvezdenec [Wed, 22 Jan 2020 00:54:44 +0000 (03:54 +0300)]
Tweak trapped rook penalty
This patch greatly increases the endgame penalty for having a trapped rook.
Idea was a result of witnessing Stockfish losing some games at CCCC exchanging
pieces in the position with a trapped rook which directly lead to a lost endgame.
This patch should partially fix such behavior making this penalty high even in
deep endgames.
Congrats! I think this might be a common pattern - scores that seem to mainly apply
to the midgame are often better with a similar (or at least fairly big) endgame value
as well. Maybe there are others eval parameters we can tweak like this...
Guenther Demetz [Thu, 16 Jan 2020 07:39:20 +0000 (08:39 +0100)]
Use (strict) greater-than-operator for 'improving'
Currently on a normal bench run in ~0,7% of cases 'improving' is set to
true although the static eval isn't improving at all, just keeping
equal. It looks like the strict gt-operator is more appropriate here,
since it returns to 'improving' its literal meaning without sideffects.
protonspring [Thu, 23 Jan 2020 17:18:58 +0000 (18:18 +0100)]
Use a std::bitset for KPKBitbase
This is a non-functional simplification. Looks like std::bitset works good
for the KPKBitbase. Thanks for Jorg Oster for helping get the speed up
(the [] accessor is faster than test()).
Speed testing: 10k calls to probe:
master 9.8 sec
patch 9.8 sec.
protonspring [Mon, 23 Dec 2019 17:58:30 +0000 (10:58 -0700)]
Determine opposite colors mathematically
This is a non-functional speed-up: master has to access SquareBB twice while this patch
determines opposite_colors just using the values of the squares. It doesn't seem to change
the overall speed of bench, but calling opposite_colors(...) 10 Million times:
master: 39.4 seconds
patch: 11.4 seconds.
The only data point I have (other than my own tests), is a quite old failed STC test:
LLR: -2.93 (-2.94,2.94) [-1.50,4.50]
Total: 24308 W: 5331 L: 5330 D: 13647
Ptnml(0-2): 315, 2577, 6326, 2623, 289
http://tests.stockfishchess.org/tests/view/5e010256c13ac2425c4a9a67
protonspring [Fri, 10 Jan 2020 22:08:47 +0000 (15:08 -0700)]
Simplify KPK classify
This is a non-functional simplification. If we use the "side to move" of the entry
instead of the template, one of the classify methods goes away. Furthermore, I've
resolved the colors in some of the statements (we're already assuming direction
using NORTH), and used stm (side to move) instead of "us," since this is much clearer
to me.
This is not tested because it is non-functional, only applies building the bitbase
and there are no changes to the binary (on my machine).
protonspring [Sat, 21 Dec 2019 22:36:29 +0000 (15:36 -0700)]
Simplify signature of remove_piece()
This is a non-functional simplification. Instead of passing the piece type
for remove_piece, we can rely on the board. The only exception is en-passant
which must be explicitly set because the destination square for the capture
is not the same as the piece to remove.
Verified also in the Chess960 castling case by running a couple of perft, see
the pull request discussion: https://github.com/official-stockfish/Stockfish/pull/2460
It is our pleasure to release Stockfish 11 to our fans and supporters.
Downloads are freely available at http://stockfishchess.org/download/
This version 11 of Stockfish is 50 Elo stronger than the last version, and
150 Elo stronger than the version which famously lost a match to AlphaZero
two years ago. This makes Stockfish the strongest chess engine running on
your smartphone or normal desktop PC, and we estimate that on a modern four
cores CPU, Stockfish 11 could give 1:1000 time odds to the human chess champion
having classical time control, and be on par with him. More specific data,
including nice cumulative curves for the progression of Stockfish strength
over the last seven years, can be found on [our progression page][1], at
[Stefan Pohl site][2] or at [NextChessMove][3].
In October 2019 Stockfish has regained its crown in the TCEC competition,
beating in the superfinal of season 16 an evolution of the neural-network
engine Leela that had won the previous season. This clash of style between an
alpha-beta and an neural-network engine produced spectacular chess as always,
with Stockfish [emerging victorious this time][0].
Compared to Stockfish 10, we have made hundreds of improvements to the
[codebase][4], from the evaluation function (improvements in king attacks,
middlegame/endgame transitions, and many more) to the search algorithm (some
innovative coordination methods for the searching threads, better pruning of
unsound tactical lines, etc), and fixed a couple of bugs en passant.
Our testing framework [Fishtest][5] has also seen its share of improvements
to continue propelling Stockfish forward. Along with a lot of small enhancements,
Fishtest has switched to new SPRT bounds to increase the chance of catching Elo
gainers, along with a new testing book and the use of pentanomial statistics to
be more resource-efficient.
Overall the Stockfish project is an example of open-source at its best, as
its buzzing community of programmers sharing ideas and daily reviewing their
colleagues' patches proves to be an ideal form to develop innovative ideas for
chess programming, while the mathematical accuracy of the testing framework
allows us an unparalleled level of quality control for each patch we put in
the engine. If you wish, you too can help our ongoing efforts to keep improving
it, just [get involved][6] :-)
Stockfish is also special in that every chess fan, even if not a programmer,
[can easily help][7] the team to improve the engine by connecting their PC to
Fishtest and let it play some games in the background to test new patches.
Individual contributions vary from 1 to 32 cores, but this year Bojun Guo
made it a little bit special by plugging a whole data center during the whole
year: it was a vertiginous experience to see Fishtest spikes with 17466 cores
connected playing [25600 games/minute][8]. Thanks Guo!
Vizvezdenec [Sun, 12 Jan 2020 23:59:06 +0000 (02:59 +0300)]
Tweak futility pruning constants
Based on recent improvement of futility pruning by @locutus2 : we lower
the futility margin to apply it for more nodes but as a compensation
we also lower the history threshold to apply it to less nodes. Further
work in tweaking constants can always be done - numbers are guessed
"by hand" and are not results of some tuning, maybe there is some more
Elo to squeeze from this part of code.
xoto10 [Sat, 11 Jan 2020 22:10:22 +0000 (22:10 +0000)]
Smarter time management near stop limit
This patch makes Stockfish search same depth again if > 60% of optimum time is
already used, instead of trying the next iteration. The idea is that the next
iteration will generally take about the same amount of time as has already been
used in total. When we are likely to begin the last iteration, as judged by total
time taken so far > 0.6 * optimum time, searching the last depth again instead of
increasing the depth still helps the other threads in lazy SMP and prepares better
move ordering for the next moves.
projects / stockfish / log