Vizvezdenec [Sun, 12 Jan 2020 23:59:06 +0000 (02:59 +0300)]
Tweak futility pruning constants
Based on recent improvement of futility pruning by @locutus2 : we lower
the futility margin to apply it for more nodes but as a compensation
we also lower the history threshold to apply it to less nodes. Further
work in tweaking constants can always be done - numbers are guessed
"by hand" and are not results of some tuning, maybe there is some more
Elo to squeeze from this part of code.
xoto10 [Sat, 11 Jan 2020 22:10:22 +0000 (22:10 +0000)]
Smarter time management near stop limit
This patch makes Stockfish search same depth again if > 60% of optimum time is
already used, instead of trying the next iteration. The idea is that the next
iteration will generally take about the same amount of time as has already been
used in total. When we are likely to begin the last iteration, as judged by total
time taken so far > 0.6 * optimum time, searching the last depth again instead of
increasing the depth still helps the other threads in lazy SMP and prepares better
move ordering for the next moves.
This updates estimates from 1.5 year ago, and adds missing terms. All estimates
from tests run on fishtest at 10+0.1 (STC), 20000 games, error bars +- 3 Elo,
see the original message in the pull request for the full list of tests.
Noteworthy changes are step 7 (futility pruning) going from ~30 to ~50 Elo
and step 13 (pruning at shallow depth) going from ~170 to ~200 Elo.
Full list of tests: https://github.com/official-stockfish/Stockfish/pull/2401
@Rocky640 made the suggestion to look at time control dependence of these terms.
I picked two large terms (early futility pruning and singular extension), so with
small relative error. It turns out it is actually quite interesting (see figure 1).
Contrary to my expectation, the Elo gain for early futility pruning is pretty time
control sensitive, while singular extension gain is not.
Figure 1: TC dependence of two search terms
Going back to the old measurement of futility pruning (30 Elo vs today 50 Elo),
the code is actually identical but the margins have changed. It seems like a nice
example of how connected terms in search really are, i.e. the value of early futility
pruning increased significantly due to changes elsewhere in search.
joergoster [Thu, 12 Dec 2019 11:53:47 +0000 (12:53 +0100)]
50-moves rule improvement for transposition table
User "adentong" reported recently of a game where Stockfish blundered a game
in a tournament because during a search there was an hash-table issue for
positions inside the tree very close to the 50-moves draw rule. This is part
of a problem which is commonly referred to as the Graph History Interaction (GHI),
and is difficult to solve in computer chess because storing the 50-moves counter
in the hash-table loses Elo in general.
Links:
Issue 2451 : https://github.com/official-stockfish/Stockfish/issues/2451
About the GHI : https://www.chessprogramming.org/Graph_History_Interaction
This patch tries to address the issue in this particular game and similar
reported games: it prevents that values from the transposition table are
getting used when the 50-move counter is close to reaching 100 (). The idea
is that in such cases values from previous searches, with a much lower 50-move
count, become less and less reliable.
More precisely, the heuristic we use in this patch is that we don't take the
transposition table cutoff when we have reached a 45-moves limit, but let the
search continue doing its job. There is a possible slowdown involved, but it will
also help to find either a draw when it thought to be losing, or a way to avoid
the draw by 50-move rule. This heuristics probably will not fix all possible cases,
but seems to be working reasonably well in practice while not losing too much Elo.
lantonov [Wed, 1 Jan 2020 08:10:39 +0000 (10:10 +0200)]
Tuned nullmove search
Tuning was done with Bayesian optimisation and sequential use of gaussian process
regressor and gaussian process classifier. The latter is used in lieu of ordinal
categorical modelling. Details will be given in Fishcooking forum topic: https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/b3uhBBJcJG4
Vizvezdenec [Fri, 3 Jan 2020 02:53:59 +0000 (05:53 +0300)]
Introduce king infiltration bonus
Add king infiltration bonus to initiative calculation. Idea is somewhat similar
to outflanking - endgames are hard to win if each king is on it side of the board.
So this adds extra bonus for one of kings crossing the middle line.
Alain SAVARD [Sat, 4 Jan 2020 18:54:35 +0000 (13:54 -0500)]
Use a faster implementation of Static Exchange Evaluation
SEE (Static Exchange Evaluation) is a critical component, so we might
indulge some tricks to make it faster. Another pull request #2469 showed
some speedup by removing templates, this version uses Ronald de Man
(@syzygy1) SEE implementation which also unrolls the for loop by
suppressing the min_attacker() helper function and exits as soon as
the last swap is conclusive.
See Ronald de Man version there:
https://github.com/syzygy1/Cfish/blob/master/src/position.c
And since we are using new SPRT statistics, and that both pull requests
finished with less than 20000 games I also tested against master as
a speed-up:
lantonov [Mon, 9 Dec 2019 18:50:47 +0000 (20:50 +0200)]
Tuned razor and futility margins
Tuning was done with Bayesian optimisation with the following parameters:
Acquisition function: Expected Improvement
alpha: 0.05
xi: 1e-4
TC: 60+0.6
Number of iterations: 100
Initial points: 5
Batch size: 20 games
For the record, the idea was to run an experimental tuning with disabled
castling in the hope to get more hits on the TrappedRook and the king in
the c1- f1-f2-c2 area
http://tests.stockfishchess.org/tests/view/5dec57be51219d7befdc76e1
A first interpretation of that tuning was green STC (0, 4) and yellow LTC (0, 4):
http://tests.stockfishchess.org/tests/view/5ded04bc51219d7befdc773a
http://tests.stockfishchess.org/tests/view/5ded1e7a51219d7befdc7760
Thank you @xoto for trying this. Indeed, because the tuned Kc2 and Kf2 values
were quite different, it was a good idea to try something more neutral.
Vizvezdenec [Sun, 8 Dec 2019 14:10:14 +0000 (17:10 +0300)]
Exclude blockers for king from mobility area
This patch excludes blockers for king from mobility area. It was tried a couple
of times by now but now it passed. Performance is not enormously good but this
patch makes a lot of sence - blockers for king can't really move until king moves
(in most cases) so logic behind it is the same as behind excluding king square
from mobility area.
Vizvezdenec [Sat, 7 Dec 2019 14:56:33 +0000 (17:56 +0300)]
Do last capture extensions for every single node
This patch simplifies latest @MJZ1977 elo gainer. Seems like PvNode check in
condition of last capture extension is not needed. Note - even if this is a
simplification it actually causes this extension to be applied more often, thus
strengthening effect of @MJZ1977's patch.
joergoster [Fri, 6 Dec 2019 09:11:45 +0000 (10:11 +0100)]
Fix output of PV lines with invalid scores #2439
As reported on the forum it is possible, on very rare occasions, that we are
trying to print a PV line with an invalid previousScore, although this line
has a valid actual score. This patch fixes output of PV lines with invalid
scores in a MultiPV search. This is a follow-up patch to 8b15961 and makes
the fix finally complete.
The reason is the i <= pvIdx condition which probably is a leftover from the
times there was a special root search function. This check is no longer needed
today and prevents PV lines past the current one (current pvIdx) to be flagged
as updated even though they do have a valid score.
xoto10 [Sun, 8 Dec 2019 11:06:19 +0000 (11:06 +0000)]
Tweak time management (failing eval)
Adjust fallingEval with score change in last 5 iterations. FallingEval adjusts
the time used on a move depending on whether the position score is better or
worse than on the previous move. This change adds a dependency on the score
change in the last 5 iterations of the current search.
Tests with original code:
STC :
LLR: 2.97 (-2.94,2.94) [-1.50,4.50]
Total: 18728 W: 4170 L: 4005 D: 10553
https://tests.stockfishchess.org/tests/view/5de68a5bb407ee7bfda68a94
the removed line is not needed, since with the conditions on SE, eval
equals ttValue (except inCheck), which must be larger than beta if the second condition
is true.
The removed line is also incorrect as eval might be VALUE_NONE at this
location if inCheck. This removal addresses part of https://github.com/official-stockfish/Stockfish/pull/2406#issuecomment-552642608
Where to go from here:
Further tests will try a similar tweak on the friendly king proximity penalty,
because recent experiments indicate that this penalty is quite sensitive,
but I wanted to try first on the larger term.
Increasing the penalty ratio to 21:8 was neutral.
http://tests.stockfishchess.org/tests/view/5de2814d727dc1d26718a671
Decreasing the penalty ratio a bit more to 9:4 seems less promising
http://tests.stockfishchess.org/tests/view/5de2f4c2727dc1d26718a691
http://tests.stockfishchess.org/tests/view/5de32ecc727dc1d26718a6b0
Vizvezdenec [Mon, 25 Nov 2019 23:56:53 +0000 (02:56 +0300)]
Simplify king danger
This patch is a cleanup/simplification of king flank defenders patch,
removing king flanks attacks linear dependance in kingdanger. Result
of experiments with quadratic kingflank defenders scaling. Rebased on
the latest master.
31m059 [Sat, 23 Nov 2019 09:03:51 +0000 (04:03 -0500)]
King danger: retire attacked-by-bishop defense
In a recent commit, "Introduce king flank defenders," a term was introduced
by Michael Chaly (@Vizvezdenec) to reduce king danger based on king defenders,
i.e., friendly attacks on our King Flank and Camp. This is a powerful idea
and broadly applicable to all of our pieces.
An earlier, but narrower, version of a similar idea was already coded into
king danger, with a term reducing king danger simply if we had a bishop and
king attacking the same square -- there is also a similar term for knights,
but roughly three times larger. I had attempted to tweak this term's coefficient
fairly recently, in a series of tests in early September which increased this
coefficient. All failed STC with significantly negative scores.
Now that the king flank defenders term has been introduced, it appears that
the bishop-defense term can be simplified away without compensation or
significant Elo loss.
Where do we go from here? This PR is a natural follow-up to "Introduce king
flank defenders," which proposed simplification with existing and overlapping
terms, such as this one. That PR also mentioned that the coefficient it
introduced appeared arbitrary, so perhaps this PR can facilitate a tweak to
increase king flank defenders' coefficient.
Additionally, this pull request is extremely similar to https://github.com/official-stockfish/Stockfish/pull/1821,
which was (coincidentally) merged a year ago, to the day (November 23, 2018).
That patch also simplified away a linear king danger tropism term, which was
soon after replaced with a quadratic term by @Vizvezdenec (which would not have
passed without the simplification). @Vizvezdenec, again by coincidence, has
recently been trying to implement a quadratic term, this time for defenders
rather than attackers. This history of this evaluation code suggests that
this simplification might be enough to help a patch for quadratic king-flank
defenders pass.
Interesting discussion on Github about this pull request:
https://github.com/official-stockfish/Stockfish/pull/2424
---
This pull request was opened less than one week before the holiday of
Thanksgiving here in the United States. In keeping with the holiday
tradition of expressing gratitude, I would like to thank our generous
CPU donors, talented forum contributors, innovative developers, speedy
fishtest approvers, and especially our hardworking server maintainers
(@ppigazzini and @tomtor). Thank you all for a year of great Stockfish
progress!
SFisGOD [Wed, 20 Nov 2019 19:31:23 +0000 (03:31 +0800)]
Simplify endgame factor for opposite colored bishops
Stockfish is continually improving. Patches that gain elo in the past may
no longer be needed as stockfish improved elsewhere. This patch removes
passed pawns count dependence in opposite colored bishops scale factor.
We used the mean of passed count pawns (~1.4) to compensate, and changed
the base value from 16 to 22.
Vizvezdenec [Wed, 20 Nov 2019 20:20:20 +0000 (23:20 +0300)]
Do lmr for more captures
Based on machinery introduced by vondele. Logic behind patch if relatively simple -
if we reduce less with high hit rate of transposition table somewhat logical is to
reduce more with low hit rate. For example enable all captures for LMR.
Threshold 0.375 is arbitrary and can be tweaked :)
Alain SAVARD [Sat, 16 Nov 2019 19:42:47 +0000 (14:42 -0500)]
Outpost Endgame values
Remove the recent rank based Outpost array by using a weighted average value
computed using a frequency analysis by rank from a large set of middle game
positions.
The higher eg values introduced by the new Outpost array (which were about
twice the previous masters) are thus preserved.
Vizvezdenec [Sun, 17 Nov 2019 18:47:17 +0000 (21:47 +0300)]
Simplify advanced pawn push pruning
This patch simplifies away all conditions related to advanced pawn pushes
in shallow depth pruning. Idea is based on fact that in master we have
advanced pawn pushes not being pruned what we are only in PV node and
when non-pawn material of opponent is > Bishop, so pretty rarely. With
this patch we will have all pruning heuristics working for this moves as
for every other move.
This patch measures how frequently search is exploring new configurations.
This is done be computing a running average of ttHit. The ttHitAverage rate
is somewhat low (e.g. 30% for startpos) in the normal case, while it can be
very high if no progress is made (e.g. 90% for the fortress I used for testing).
This information can be used to influence search. In this patch, by adjusting
reductions if the rate > 50%. A first version (using a low ttHitAverageResolution
and this 50% threshold) passed testing:
However, as discussed in pull request 2414, using a larger ttHitAverageResolution
gives a better approximation of the underlying distributions. This needs a slight
adjustment for the threshold as the new distributions are shifted a bit compared
to the older ones, and this threshold seemingly is sensitive (we used 0.53125 here).
https://github.com/official-stockfish/Stockfish/pull/2414
This final version also passed testing, and is used for the patch:
Vizvezdenec [Sat, 16 Nov 2019 11:53:11 +0000 (14:53 +0300)]
Introduce king flank defenders
This patch implements what we have been trying for quite some time -
dependance of kingdanger on balance of attackers and defenders of king
flank, to avoid overestimate attacking power if the opponent has enough
defenders of king position. We already have some form of it in bishop
and knight defenders - this is further work in this direction.
What to do based on this?
1) constant 4 is arbitrary, maybe it is not optimal
2) maybe we can use quadratic formula as in kingflankattack
3) simplification into alrealy existing terms is always a possibility :)
4) overall kingdanger tuning always can be done.
SFisGOD [Tue, 12 Nov 2019 01:22:09 +0000 (09:22 +0800)]
Rank-based outposts
Introduce OutpostRank[RANK_NB] which contains a bonus according to
the rank of the outpost. We use it for the primary Outpost bonus.
The values are based on the trends of the SPSA tuning run with some
manual tweaks.
The inspiration for this patch came from Stefan Geschwentner's attempt
of modifying BishopPawns into a rank-based penalty. Michael Stembera
suggested that maybe the S(0, 0) ranks (3rd, 7th and also maybe 8th)
can still be tuned. This would expand our definition of Outpost and
OutpostRanks would be removed altogether. Special thanks to Mark Tenzer
for all the help and excellent suggestions.
Miguel Lahoz [Sun, 10 Nov 2019 09:49:06 +0000 (17:49 +0800)]
Shallow depth pruning on NonPV advanced pawn push
Usually advanced pawn pushes are not considered in shallow depth pruning
because it is risky to do so with possible promotions near the horizon.
However, this heuristic is not also beneficial on NonPV nodes since we
can afford to take slightly more risk on less important nodes.
The removed lines approximately duplicate equivalent logic in the movePicker.
Adjust the futility_move_count to componsate for some difference
(the movePicker prunes one iteration of the move loop later).
It would be interesting to see if we can extract some Elo gain
from the new futility_move_count formula, for instance by somehow
incorporating the final -1 in the 5 constant, or adding a linear
term to the quadratics...
Alain SAVARD [Sun, 27 Oct 2019 20:16:26 +0000 (16:16 -0400)]
Sequencing tweak in tbprobe()
Followup of "issue" #2372, which was in fact a small speed-up
proposal by user @d3vv for the probing code of tablebases.
See comments on this issue where it was proven by Alin Savard that the
proposed change is more efficient on average than master on all type of
sequences it will usually be called.
Note that on gcc 4.3, this will produce a bogus warning which was solved
with ulterior gcc versions: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=43949
```
setoption name Hash value 8
setoption name Threads value 1
position fen 8/1p2KP2/1p4q1/1Pp5/2P5/N1Pp1k2/3P4/1N6 b - - 76 40
go depth 49
```
even though the position is a draw. Generally, SF tries to display only
proven mate scores, so this is a bug.
This was posted http://www.talkchess.com/forum3/viewtopic.php?f=2&t=72166
by Uri Blass, with the correct analysis that this must be related to the
50 moves draw rule being ignored somewhere.
Indeed, this is possible as positions and there eval are stored in the TT,
without reference to the 50mr counter. Depending on the search path followed
a position can thus be mate or draw in the TT (GHI or Graph history interaction).
Therefore, to prove mate lines, the TT content has to be used with care. Rather
than ignoring TT content in general or for mate scores (which impact search or
mate finding), it is possible to be more selective. In particular, @WOnder93
suggested to only ignore the TT if the 50mr draw ply is closer than the mate
ply. This patch implements this idea, by clamping the eval in the TT to
+-VALUE_MATED_IN_MAX_PLY. This retains the TTmove, but causes a research of
these lines (with the current 50mr counter) as needed.
This patch hardly ever affects search (as indicated by the unchanged
bench), but fixes the testcase. As the conditions are very specific,
also mate finding will almost never be less efficient (testing welcome).
It was also shown to pass STC and LTC non-regression testing, in a form
using if/then/else instead of ternary operators:
This reverts the previous commit. The PSQT changes in this previous
commit originated from tests against quite an old version of master
which did not include the other PSQT changes of 474d133 for the other
pieces, and there might be some unknown interactions between the PSQT
tables. So we made a non-regression test of the last commit against the
last-but-one commit. This test failed, leading to the revert decision.
SFisGOD [Mon, 4 Nov 2019 19:06:41 +0000 (03:06 +0800)]
Rook PSQT Tuned
This patch uses about half the changes of the SPSA tuning run:
http://tests.stockfishchess.org/tests/view/5dba93d30ebc5925b64ed3bf
About a month ago, xoto10's patch raised the mg value of the third rank
center files from -1 to 7 to encourage rook lifts to the third rank. About
three days later, Rocky's patch lowered this value from 7 to 3. This patch
raises that again from 3 to 12 and ends up greater than the original rook
lift patch.
MichaelB7 [Sun, 3 Nov 2019 00:04:05 +0000 (20:04 -0400)]
Remove shuffle extension
It was noted in an earlier patch that all of the positions below needed the
Shuffle Detection idea to be solved:
3r4/p3r1pk/PpBb1pRp/1KpPpP1P/2P1P1R1/8/8/8 b - - 32 86
8/8/8/1k6/2p5/p1K5/N2B2r1/8 b - - 59 109
1r4k1/1r1bq3/4p1p1/3pPpPp/pNpN1P1P/P1PnQ3/1PK5/1R3R2 b - - 13 82
5k2/3b4/5p2/p1p1pPp1/PpPpP1Pp/1P1P3P/8/3R1K2 w - - 20 1
But Stockfish has envolved a bit since the Shuffle Detection patch introduction,
and this patch proves Stockfish is able to solves these drawn positions without it,
even on single core without EGTB.
Example of search by Michael Byrne for the FEN position:
q1B5/1P1q4/8/8/8/6R1/8/1K1k4 w - - 0 1
This position is win for white and the only moves that wins is Rg1 - all other moves
either draw or lose. With single core and 1024M hash, it is solved without shuffle
detection in 38 seconds on my machine (with no EGTB). This was the position that was
locked in a loop in the initial shuffle detection patch!
31m059 [Fri, 1 Nov 2019 04:27:19 +0000 (00:27 -0400)]
Make Square and Bitboard operators commutative
As Stockfish developers, we aim to make our code as legible and as close
to simple English as possible. However, one of the more notable exceptions
to this rule concerns operations between Squares and Bitboards.
Prior to this pull request, AND, OR, and XOR were only defined when the
Bitboard was the first operand, and the Square the second. For example,
for a Bitboard b and Square s, "b & s" would be valid but "s & b" would not.
This conflicts with natural reasoning about logical operators, both
mathematically and intuitively, which says that logical operators should
commute.
More dangerously, however, both Square and Bitboard are defined as integers
"under the hood." As a result, code like "s & b" would still compile and give
reasonable bench values. This trap occasionally ensnares even experienced
Stockfish developers, but it is especially dangerous for new developers not
aware of this peculiarity. Because there is no compilation or runtime error,
and a reasonable bench, only a close review by approvers can spot this error
when a test has been submitted--and many times, these bugs have slipped past
review. This is by far the most common logical error on Fishtest, and has
wasted uncountable STC games over the years.
However, it can be fixed by adding three non-functional lines of code. In this
patch, we define the operators when the operands are provided in the opposite
order, i.e., we make AND, OR, and XOR commutative for Bitboards and Squares.
Because these are inline methods and implemented identically, the executable
does not change at all.
This patch has the small side-effect of requiring Squares to be explicitly
cast to integers before AND, OR, or XOR with integers. This is only performed
twice in Stockfish's source code, and again does not change the executable at
all (since Square is an enum defined as an integer anyway).
For demonstration purposes, this pull request also inverts the order of one AND
and one OR, to show that neither the bench nor the executable change. (This
change can be removed before merging, if preferred.)
I hope that this pull request significantly lowers the barrier-of-entry for new
developer to join the Stockfish project. I also hope that this change will improve
our efficiency in using our generous CPU donors' machines, since it will remove
one of the most common causes of buggy tests.
Following helpful review and comments by Michael Stembera (@mstembera), we add
a further clean-up by implementing OR for two Squares, to anticipate additional
traps developers may encounter and handle them cleanly.
protonspring [Thu, 31 Oct 2019 15:01:33 +0000 (09:01 -0600)]
Simplify kingRing
Simplify the king ring initialization and make it more regular, by just
moving the king square off the edges and using PseudoAttacks by king from
this new square.
There is a small functional difference from the previous master, as the
old master excludes the original ksq square while this patch always includes
the nine squares block (after moving the king from the edges). Additionally,
master does not adjust the kingRing down if we are on relative rank 8,
while this patch treats all of the edges the same.
protonspring [Thu, 31 Oct 2019 16:17:46 +0000 (17:17 +0100)]
Consolidate pawn_push and up
This is a non-functional simplification. Pawn_push and Up are redundant.
If we make up pawn_push, we can use it for all of the Up's and Down's.
In this version, I've also left the Up and Down constants so that there
is no worse readability.
Make dynamic contempt weight factor dependent on static contempt so that higher
static contempt implies less dynamic contempt and vice versa. For default contempt
24 this is a non-functional change. But tests with contempt 0 shows an elo gain.
Also today is my birthday so i have already give to myself a gift with this patch :-)!
Further proceedings:
in the past we checked for default contempt that it doesn't regress against
contempt 0. Now that the later is stronger and the former is the same strength
this should be rechecked. Perhaps the default contempt have to be lowered.
It would be interesting to get some idea of the impact of this patch outside
of the 0-24 contempt range.
This PR refactors update_quiet_stats, update_capture_stats and search to more clearly reflect what is actually done.
Effectively, all stat updates that need to be done after search is finished and a bestmove is found,
are collected in a new function ```final_stats_update()```. This shortens our main search routine, and simplifies ```update_quiet_stats```.
The latter function is now more easily reusable with fewer arguments, as the handling of ```quietsSearched``` is only needed in ```final_stats_update```.
```update_capture_stats```, which was only called once is now integrated in ```final_stats_update```, which allows for removing a branch and reusing some ```stat_bonus``` calls. The need for refactoring was also suggested by the fact that the comments of ```update_quiet_stats``` and ```update_capture_stats``` were incorrect (e.g. ```update_capture_stats``` was called, correctly, also when the bestmove was a quiet and not a capture).
- Cleanups by Alain
- Group king attacks and king defenses
- Signature of futility_move_count()
- Use is_discovery_check_on_king()
- Simplify backward definition
- Use static asserts in move generator
- Factor a statement in move generator
Current master employs a scheme to adjust reductions on singular
nodes that is somewhat controversial, see
https://github.com/official-stockfish/Stockfish/pull/2167
This patch removes this use of a search result outside of [a,b],
by observing that the main effect of this code is to adjust the
reduction by an average of ~2 (1.7) rather than 1.
Stockfish crashes immediately if users enter a wrong file name (or even an existing
folder name) for debug log file. It may be hard for users to find out since it prints
nothing. If they enter the string via a chess GUI, the chess GUI may remember and
auto-send to Stockfish next time, makes Stockfish crashes all the time. Bug report by
Nguyen Hong Pham in this issue: https://github.com/official-stockfish/Stockfish/issues/2365
This patch avoids the crash and instead prefers to exit gracefully with a error
message on std:cerr, like we do with the fenFile for instance.
With the current questions and issues around threading, I had a look at
https://github.com/official-stockfish/Stockfish/issues/2299.
It seems there was a problem with data races when requesting eval via UCI while
a search was already running. To fix this an extra thread uithread was created,
presumably to avoid an overlap with Threads.main() that was causing problems.
Making this eval request seems to be outside the scope of UCI, and @vondele also
reports that the data race is not even fixed reliably by this change. I suggest
we simplify the threading here by removing this uithread and adding a comment
signaling that user should not request eval when a search is already running.
The current bench is missing a position with high 50 moves rule counter,
making most 'shuffle' tests based on 50mr > N seem non-functional.
This patch adds one FEN with high 50mr counter to address this issue
(taken from a recent tcec game).
Four new FENs:
- position with high 50mr counter
- tactical position with many captures, checks, extensions, fails high/low
- two losses by Stockfish in the S16 bonus games against Houdini
See the pull request for nice comments by @Alayan-stk-2 about each position
in bench: https://github.com/official-stockfish/Stockfish/pull/2338
31m059 [Mon, 7 Oct 2019 18:47:43 +0000 (14:47 -0400)]
No reachable outpost bonus for bishops
Previously, we used various control statements and ternary operators to divide
Outpost into four bonuses, based on whether the outpost was for a knight or
bishop, and whether it was currently an Outpost or merely a potential ("reachable")
one in the future. Bishop outposts, however, have traditionally been worth far
less Elo in testing. An attempt to remove them altogether passed STC, but failed LTC.
Here we include a narrower simplification, removing the reachable Outpost bonus
for bishops. This bonus was always suspect, given that its current implementation
conflicts directly with BishopPawns. BishopPawns penalizes our bishops based on the
number of friendly pawns on the same color of square, but by definition, Outposts
must be pawn-protected! This PR helps to alleviate this conceptual contradiction
without loss of Elo and with slightly simpler code.
On a code level, this allows us to simplify a ternary operator into the previous
"if" block and distribute a multiplication into an existing constant Score. On a
conceptual level, we retire one of the four traditional Outpost bonuses.
Alayan [Mon, 7 Oct 2019 17:02:33 +0000 (19:02 +0200)]
Adjust aspiration window with eval
This patch changes the base aspiration window size depending on the absolute
value of the previous iteration score, increasing it away from zero. This
stems from the observation that the further away from zero, the more likely
the evaluation is to change significantly with more depth. Conversely, a
tighter aspiration window is more efficient when close to zero.
A beneficial side-effect is that analysis of won positions without a quick
mate is less prone to waste nodes in repeated fail-high that change the eval
by tiny steps.
Future work : the values used in this patch were only a reasonable guess.
Further testing should unveil more optimal values. However, the aspiration
window is rather tight with a minimum of 21 internal units, so discrete
integers put a practical limitation to such tweaking.
More exotic experiments around the aspiration window parameters could also
be tried, but efficient conditions to adjust the base aspiration window size
or allow it to not be centered on the current evaluation are not obvious.
The aspiration window increases after a fail-high or a fail-low is another
avenue to explore for potential enhancements.
Alain SAVARD [Sun, 6 Oct 2019 22:48:19 +0000 (00:48 +0200)]
Adjust pawn span
Run as a simplification
a) insures that pawn attacks are always included in the pawn span
(this "fixes" the case where some outpost or reachable outpost
bonus were awarded on squares controlled by enemy pawns).
b) compute the full span only if not "backward" or not "blocked".
By looking at "blocked" instead of "opposed", we get a nice simpli-
fication and the "new" outpost detection is almost identical, except
a few borderline cases on rank 4.
Ondrej Mosnacek [Sun, 6 Oct 2019 07:57:20 +0000 (09:57 +0200)]
Make priorCapture a bool
It is always used as a bool, so let's make it a bool straight away.
We can always redefine it as a Piece in a later patch if we want
to use the piece type or the piece color.
VoyagerOne [Sat, 5 Oct 2019 14:42:36 +0000 (10:42 -0400)]
Introduce separate counter-move tables for captures
Enhance counter-move history table by adding a capture/no-capture dimension,
depending wether the previous move was a quiet move or a capture. This doubles
the size of the table but provides more accurate move ordering.
Brian Sheppard [Sat, 28 Sep 2019 20:27:23 +0000 (16:27 -0400)]
Eliminate ONE_PLY
Simplification that eliminates ONE_PLY, based on a suggestion in the forum that
support for fractional plies has never been used, and @mcostalba's openness to
the idea of eliminating it. We lose a little bit of type safety by making Depth
an integer, but in return we simplify the code in search.cpp quite significantly.
No functional change
------------------------------------------
The argument favoring eliminating ONE_PLY:
* The term “ONE_PLY” comes up in a lot of forum posts (474 to date)
https://groups.google.com/forum/?fromgroups=#!searchin/fishcooking/ONE_PLY%7Csort:relevance
* There is occasionally a commit that breaks invariance of the code
with respect to ONE_PLY
https://groups.google.com/forum/?fromgroups=#!searchin/fishcooking/ONE_PLY%7Csort:date/fishcooking/ZIPdYj6k0fk/KdNGcPWeBgAJ
* To prevent such commits, there is a Travis CI hack that doubles ONE_PLY
and rechecks bench
* Sustaining ONE_PLY has, alas, not resulted in any improvements to the
engine, despite many individuals testing many experiments over 5 years.
The strongest argument in favor of preserving ONE_PLY comes from @locutus:
“If we use par example ONE_PLY=256 the parameter space is increases by the
factor 256. So it seems very unlikely that the optimal setting is in the
subspace of ONE_PLY=1.”
There is a strong theoretical impediment to fractional depth systems: the
transposition table uses depth to determine when a stored result is good
enough to supply an answer for a current search. If you have fractional
depths, then different pathways to the position can be at fractionally
different depths.
In the end, there are three separate times when a proposal to remove ONE_PLY
was defeated by the suggestion to “give it a few more months.” So… it seems
like time to remove this distraction from the community.
See the pull request here:
https://github.com/official-stockfish/Stockfish/pull/2289
mstembera [Sat, 24 Aug 2019 22:04:41 +0000 (15:04 -0700)]
Remove temporary shelter array
Remove temporary array of shelters and avoid iterating over it each time to find
if the shelter values after castling are better than the current value.
Work done on top of https://github.com/official-stockfish/Stockfish/pull/2277
Tweak the pawn attack span for backward pawns and the zone behind
opponent opposing pawns. This is important in positional play and
one of weaknesses of the engine in recent high level games.
In lazySMP it makes sense to prune a little more, as multiple threads
search wider. We thus increase the prefactor of the reductions slowly
as a function of the threads. The prefactor of the log(threads) term
is a parameter, this pull request uses 1/2 after testing.
Note:
A larger prefactor (1) passed similar tests at STC and LTC (8 threads),
while a very large one (2) passed STC quickly but failed LTC (8 threads).
For the single-threaded case there is no functional change.
Extend castling independently of singular extension
A curious feature of Stockfish's current extension code is its repeated
use of "else if." In most cases, this makes no functional difference,
because no more than one extension is applied; once one extension has
been applied, the remaining ones can be safely ignored.
However, if most singular extension search conditions are true, except
"value < singularBeta", no non-singular extensions (e.g., castling) can
be performed!
Three tests were submitted, for three of Stockfish's four non-singular
extensions. I excluded the shuffle extension, because historically there
have been concerns about the fragility of its conditions, and I did not
want to risk causing any serious search problems.
- Modifying the passed pawn extension appeared roughly neutral at STC. At
best, it appeared to be an improvement of less than 1 Elo.
- Modifying check extension performed very poorly at STC
- Modifying castling extension (this patch) produced a long "yellow" run
at STC (insufficient to pass, but positive score) and a strong LTC.
In simple terms, prior to this patch castling extension was occasionally
not applied during search--on castling moves. The effect of this patch is
to perform castling extension on more castling moves. It does so without
adding any code complexity, simply by replacing an "else if" with "if" and
reordering some existing code.
- It seems strange to me that check extension performed so poorly -- clearly
some of the singular extension conditions are also very important for check
extension. I am not an expert in search, and I do not have any intuition
about which of the eight conditions is/are the culprit. I will try a
succession of eight STC tests to identify the relevant conditions, then try
to replicate this PR for check extension.
- Recent tests interacting with the castle extension may deserve retesting.
I will shortly resubmit a few of my recent castling extension tweaks, rebased
on this PR/commit.
My deepest thanks to @noobpwnftw for the extraordinary CPU donation, and to
all our other fishtest volunteers, who made it possible for a speculative LTC
to pass in 70 minutes!
Remove the RookOnPawn logic (for rook on rank 5 and above aligning with pawns
on same row or file) which was overlapping with a few other parameters.
Inspired by @31m059 interesting result hinting that a direct attack on pawns
instead of PseudoAttacks might work.
http://tests.stockfishchess.org/tests/view/5d89a7c70ebc595091801b8d
After a few attempts by me and @31m059, and some long STC greens but red LTC,
as a proof of concept I first tried a local SPSA at VSTC trying to tune related
rook psqt rows, and mainly some rook related stuff in evaluate.cpp.
Result was STC green, but still red LTC,
Finally a 100M fishtest SPSA at LTC proved successful both at STC and LTC.
All this was possible with the awesome fishtest contributors.
At some point, I had 850 workers on the last test !
As part of the investigation of the hang caused by an incorrect implementation
of condition_variable in libwinpthread, it was realized that our custom Mutex
implementation is no longer needed. Prior to lazySMP this custom implementation
resulted in a 30% speedup, but now no speed difference can be measured as no
mutex is used on the hot path in lazySMP.
Restore development version (revert previous commit)
Revert the previous patch now that the binary for the super-final
of TCEC season 16 has been sent.
Maybe the feature of showing the name of compiler will be added to the
master branch in the future. But we may use a cleaner way to code it, see
some ideas using the Makefile approach at the end of pull request #2327 :
https://github.com/official-stockfish/Stockfish/pull/2327
This patch changes the weight for counting supports of pawns
from 17 to 21. Hopefully Stockfish will accept to play a bit
more of closed or semi-closed positions.
Thanks to @snicolet for mentioning that Komodo does this a lot and
Stockfish doesn't, which gave me the idea for this patch, and to
@noobpwnftw for providing cores to fishtest which allowed very quick
testing.
Future work: perhaps this can be refined somehow to encourage this
on other files, my attempts have failed.
As in that case, this test was yellow at STC and LTC, but green at VLTC,
indicative of the fact that the higher the drawrate, the more likely this
idea is beneficial.
It seems there is no other way to specify stack size on std::thread than linker
flags and the effective flags are named differently in many toolchains. On
toolchains where pthread is always available, this patch changes the stack
size change in our C++ code via pthread to ensure a minimum stack size of 8MB,
instead of relying on linker defaults which may be platform-specific.
Also raises default stack size on OSX to current Linux default (8MB) just to
be safe.
This patch decreases the endgame scale factor using the 50 moves counter.
Looking at some games with this patch, it seems to have two effects on
the playing style:
1) when no progress can be made in late endgames (for instance in fortresses
or opposite bishops endgames) the evaluation will be largely tamed down
towards a draw value.
2) more interestingly, there is also a small effect in the midgame play because
Stockfish will panic a little bit if there are more than four consecutive
shuffling moves with an advantage: the engine will try to move a pawn or to
exchange a piece to keep the advantage, so the follow-ups of the position
will be discovered earlier by the alpha-beta search.
This patch finally introduces something that was tried for years: midgame score
dependance on complexity of position. More precisely, if the position is very
simplified and the complexity measure calculated in the initiative() function
is inferior to -50 by an amount d, then we add this value d to the midgame score.
One example of play of this patch will be (again!) 4 vs 3 etc same flank endgames
where sides have a lot of non-pawn material: 4 vs 3 draw mostly remains the same
draw even if we add a lot of equal material to both sides.
STC run was stopped after 200k games (and not converging):
LLR: -1.75 (-2.94,2.94) [0.50,4.50]
Total: 200319 W: 44197 L: 43310 D: 112812
http://tests.stockfishchess.org/tests/view/5d7cfdb10ebc5902d386572c
This is the first and not really precise version, a lot of other stuff can be
tried on top of it (separate complexity for middlegame, some more terms, even
simple retuning of values).
The recently-added RookOnQueenFile evaluation term (36e4a86) provided a bonus
for placing our rook on the same file as an enemy queen.
Here, we relax a condition in this bonus, broadening its effect to any queen.
It is also strategically desirable to place the rook on the same file as a friendly
queen, so the restriction on the queen's color is removed.
Follow-up to previous commit. Update the documentation for the user when using `make`,
to show the preferred bmi2 compile in the advanced examples section.
Note: I made a mistake in the previous commit comment, the documentation is shown when
using `make` or `make help`, not `make --help`.
The only change done to the Makefile to get a somewhat faster binary as
discussed in #2291 is to add -msse4 to the compile options of the bmi2 build.
Since all processors supporting bmi2 also support sse4 this can be done easily.
It is a useful step to avoid sending around custom and poorly tested builds.
The speedup isn't enough to pass [0,4] but it is roughly 1.15Elo and a LOS of 90%:
LLR: -2.95 (-2.94,2.94) [0.00,4.00]
Total: 93009 W: 20519 L: 20316 D: 52174
Also rewrite the documentation for the user when using `make --help`, so that
the order of architectures for x86-64 has the more performant build one on top.
Scale down complexity for almost unwinnable endgames
This patch greatly scales down complexity of endgames when the
following conditions are all true together:
- pawns are all on one flank
- stronger side king is not outflanking weaker side
- no passed pawns are present
This should improve stockfish evaluation of obvious draws 4 vs 3, 3 vs 2
and 2 vs 1 pawns in rook/queen/knight/bishop single flank endgames where
strong side can not make progress.
a) This could be a powerful idea for refining some parts of the evaluation
function, a bit like when we try quadratics or other equations to emphasize
certain situations (xoto10).
b) Some other combinaison values for this bonus can be done further, or
overall retuning of weight and offset while keeping the formula simple.
This is a simplification that integrated WeakLever into doubled pawns.
Since we already check for !support for Doubled pawns, it is trivial
to check for weak lever by just checking more_than_one(lever).
We also introduce the Score * bool operation overload to remove some
casts in the code.
Maintain best move counter at the root and allow there only moves which has a counter
of zero for Late Move Reduction. For compensation only the first three moves are excluded
from Late Move Reduction per default instead the first four moves.
What we can further do:
- here we use a simple counting scheme but perhaps some aging to fade out early iterations
could be helpful
- use the best move counter also at inner nodes for LMR and/or pruning
protonspring [Sat, 24 Aug 2019 06:16:20 +0000 (08:16 +0200)]
Improve signature of evaluate_shelter()
Remove one parameter in function evaluate_shelter(), making all
comparisons for castled/uncastled shelter locally in do_king_safety().
Also introduce BlockedStorm penalty.
Passed non-regression test at STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 65864 W: 14630 L: 14596 D: 36638
http://tests.stockfishchess.org/tests/view/5d5fc80c0ebc5939d09f0acc
protonspring [Mon, 12 Aug 2019 14:42:28 +0000 (08:42 -0600)]
Consolidate CastlingSide and CastlingRights
This is a non-functional simplification that removes CastlingSide and
implements the functionality in CastlingRights (thanks to Jörg Oster
for a comment on the first version of this patch).
projects / stockfish / log