this patch proposes to simplify back to this basic and easier to
understand version. In case there is a need to run a [-3, 1] VLTC on
this one, it can be done, but it is resource intensive, and not needed
IMO.
Looking at this 2 ideas being recent clean elo gainers I have a feeling that we can add also rook and queen protection bonuses or overall move this stuff in pieces loop in the same way as we do pieces attacking bonuses on their kingring... :) Thx fisherman for original idea.
The "move" class variable is Movepick is removed (removes some abstraction) which saves a few assignment operations, and the effects of "filter" is limited to the current move (movePtr). The resulting code is a bit more verbose, but it is also more clear what is going on. This version is NOT tested, but is substantially similar to:
We can remove the values in Pawns if we just use the piece arrays in Position. This reduces the size of a pawn entry. This simplification passed individually, and in concert with ps_passedcount100 (removes passedCount storage in pawns.).
This is very dangerous code by itself because triggers **at the leafs**
and in the above position keeps extending endlessly. In normal games
time deadline makes the search to stop sooner or later, but in fixed
seacrch we just hang possibly for a very long time. This is not acceptable
because 'go depth 13' shall not be a surprise for any position.
Introduce a new search extension when pushing an advanced passed pawn is
also suggested by the first killer move. There have been previous tests
which have similar ideas, mostly about pawn pushes, but it seems to be
overkill to extend too many moves. My idea is to limit the extension to
when a move happens to be noteworthy in some other way as well, such as
in this case, when it is also a killer move.
For future tests, it looks like this will interact heavily with passed
pawn evaluation. It may be good to try more variants of some of the more
promising evaluations tests/tweaks.
Instead of looping through kfrom,kto, rfrom, rto, we can use BetweenBB. This is less lines of code and it is more clear what castlingPath actually is. Personal benchmarks are all over the place. However, this code is only executed when loading a position, so performance doesn't seem that relevant.
Raise kingDanger threshold and adjust constant term #2087
The kingDanger term is intended to give a penalty which increases rapidly in the middlegame but less so in the endgame. To this end, the middlegame component is quadratic, and the endgame component is linear. However, this produces unintended consequences for relatively small values of kingDanger: the endgame penalty will exceed the middlegame penalty. This remains true up to kingDanger = 256 (a S(16, 16) penalty), so some of these inaccurate penalties are actually rather large.
In this patch, we increase the threshold for applying the kingDanger penalty to eliminate some of this unintended behavior. This was very nearly, but not quite, sufficient to pass on its own. The patch was finally successful by integrating a second kingDanger tweak by @Vizvezdenec, increasing the kingDanger constant term slightly and improving both STC and LTC performance.
Where do we go from here? I propose that in the future, any attempts to tune kingDanger coefficients should also consider tuning the kingDanger threshold. The evidence shows clearly that it should not be automatically taken to be zero.
Special thanks to @Vizvezdenec for the kingDanger constant tweak. Thanks also to all the approvers and CPU donors who made this possible!
Marco Costalba [Sat, 6 Apr 2019 10:43:41 +0000 (12:43 +0200)]
Fix sed for OS X (#2080)
The sed command is a bit different in Mac OS X (why not!).
The ‘-i’ option required a parameter to tell what extension to add for the
backup file. To fix it, just add extension for backup file, for example ‘.bak’
Use average bestMoveChanges across all threads #2072
The current update only by main thread depends on the luck of
whether main thread sees any/many changes to the best move or not.
It then makes large, lumpy changes to the time to be
used (1x, 2x, 3x, etc) depending on that sample of 1.
Use the average across all threads to get a more reliable
number with a smoother distribution.
Further work:
Similar changes might be possible for the fallingEval and timeReduction calculations (and elsewhere?), using either the min, average or max values across all threads.
protonspring [Mon, 25 Mar 2019 19:04:14 +0000 (13:04 -0600)]
Use simple array for Pawns Connected bonus #2061
Simplification which removes the pawns connected array.
Instead of storing the values in an array, the values are
calculated real-time. This is about 1.6% faster on my machines.
Moez Jellouli [Sun, 31 Mar 2019 08:51:08 +0000 (10:51 +0200)]
Shuffle detection #2064
Shuffle detection procedure :
Shuffling positions are detected if
the last 36 moves are reversible (rule50_count() > 36),
the position have been already in the TT,
there is a still a pawn on the board (to avoid special endings like KBN vs K).
The position is then judged as a draw.
An extension is realized if we already made 14 successive reversible moves in PV to accelerate the detection of the eventual draw.
To go further : we can still improve the idea. The length of the tests need a lot of ressources.
the limit of 36 is logic but must be checked again for special zugzwang positions,
this limit can be decreased in special positions,
the limit of 14 moves for extension has not been tuned.
protonspring [Sun, 31 Mar 2019 08:43:20 +0000 (02:43 -0600)]
Accessor for SquareBB #2067
This is a non-functional code style change.
If we add an accessor function for SquareBB we can consolidate all of the asserts. This is also a bit cleaner because all SquareBB accesses go through this method making future changes easier to manage.
This is covered by the line just before. If we would like to protect
against the piece value of e.g. a N == B, this could be done by an
assert, no need to do this at runtime.
protonspring [Sun, 24 Mar 2019 16:37:38 +0000 (10:37 -0600)]
Simplify Passed Pawns (#2058)
This is a non-functional simplification/speedup.
The truth-table for popcount(support) >= popcount(lever) - 1 is:
------------------lever
------------------0-------1---------2
support--0------X-------X---------0
-----------1------X-------X---------X
-----------2------X-------X---------X
Thus, it is functionally equivalent to just do: support || !more_than_one(lever) which removes the expensive popcounts and the -1.
Result of 20 runs:
base (...h_master.exe) = 1451680 +/- 8202
test (./stockfish ) = 1454781 +/- 8604
diff = +3101 +/- 931
xoto10 [Sun, 10 Mar 2019 21:57:48 +0000 (21:57 +0000)]
Remove !extension check #2045
While looking at pruning using see_ge() (which is very valuable)
it became apparent that the !extension test is not adding any
value - simplify it away.
Further work could be to optimize the remaining see_ge() test. The idea of less pruning at higher depths is valuable, but perhaps the test (-PawnValueEg * depth) can be improved.
CoffeeOne [Wed, 20 Mar 2019 13:50:41 +0000 (14:50 +0100)]
Skip skipping thread scheme (#1972)
Several simplification tests (all with the bounds [-3,1]) were run:
5+0.05 8 threads, failed very quickly:
http://tests.stockfishchess.org/tests/view/5c439a020ebc5902bb5d3970
20+0.2 8 threads, also failed, but needed a lot more games:
http://tests.stockfishchess.org/tests/view/5c44b1b70ebc5902bb5d4e34
Marco Costalba [Tue, 12 Mar 2019 07:35:10 +0000 (08:35 +0100)]
Increase thread stack for OS X (#2035)
On OS X threads other than the main thread are created with a reduced stack
size of 512KB by default, this is dangerously low for deep searches, so
adjust it to TH_STACK_SIZE. The implementation calls pthread_create() with
proper stack size parameter.
Verified for no regression at STC enabling the patch on all platforms where
pthread is supported.
protonspring [Sun, 3 Mar 2019 14:53:36 +0000 (07:53 -0700)]
Shrink Reductions[] array to one dimension
This is a non-functional patch which shrinks the reductions array.
This saves about 8Kb of memory.
The only slow part of master's reductions array is the calculation
of the log values, so using a separate array for those values and
calculating the rest real-time appears to be just as fast as master.
This removes the skipQuiets variable, as was done in an earlier round by
@protonspring, but fixes an oversight which led to wrong mate
announcements. Quiets can only be pruned when there is no mate score, so
set moveCountPruning at the right spot.
tested as a fix at STC:
LLR: 2.95 (-2.94,2.94) [-3.00,1.00]
Total: 66321 W: 14690 L: 14657 D: 36974
http://tests.stockfishchess.org/tests/view/5c74f3170ebc5925cffd4b3c
and as the full patch at LTC:
LLR: 2.96 (-2.94,2.94) [-3.00,1.00]
Total: 25903 W: 4341 L: 4228 D: 17334
http://tests.stockfishchess.org/tests/view/5c7540030ebc5925cffd506f
Kurtbusch [Fri, 4 Jan 2019 23:27:14 +0000 (18:27 -0500)]
Add KNNvKP Endgame Heuristic
This is a somewhat different patch. It fixes blindspots for
two knights vs pawn endgame.
With local testing starting from random KNNvKP positions where the
pawn has not advanced beyond the 4th rank (thanks @protonspring !)
at 15+0.15 (4 cores), this went +105=868-27 against master. All except
two losses were won in reverse.
The heuristic is simple but effective - the strategy in these endgames
is to push the opposing king to the corner, then move the knight that's
blocking the pawn in for the checkmate while the pawn is free to move
and prevents stalemate. This patch gives SF the little boost it needs
to search the relevant king-cornering mating lines.
See the discussion in pull request 1939 for some more good results for
this test in independant tests:
https://github.com/official-stockfish/Stockfish/pull/1939
protonspring [Wed, 13 Feb 2019 21:58:43 +0000 (14:58 -0700)]
Remove PvNode dimension from Reductions array
This is a functional simplification: if we simply subtract one to Reductions[]
when PvNode is set, we can remove this dimension of the multidimensional array.
I think this saves about 8K of memory.
protonspring [Thu, 14 Feb 2019 21:23:24 +0000 (14:23 -0700)]
Remove skipQuiets variable in search()
This is a functional simplification. The moveCountPruning variable and the
skipQuiets variable are similar enough in function that they can be combined.
This removes the skipQuiets variable in search.
protonspring [Sun, 13 Jan 2019 02:30:56 +0000 (19:30 -0700)]
Change outposts to single value #1946
This is a functional simplification of the Outposts array
moving it to a single value. This is a duplicate PR because
I couldn't figure out how to fix the original one.
The idea is from @31m059 with formatting recommendations by @snicolet.
protonspring [Fri, 8 Feb 2019 08:54:38 +0000 (01:54 -0700)]
Remove Some Bitboard Arrays (#1963)
This is non-functional. These 5 arrays are simple enough to calculate real-time and maintaining an array for them does not help. Decreases the memory footprint.
This seems a tiny bit slower on my machine, but passed STC well enough. Could someone verify speed?
Miguel Lahoz [Tue, 29 Jan 2019 14:26:03 +0000 (22:26 +0800)]
Extend discovered checks regardless of SEE
A simple idea, but it makes sense: in current master the search is extended
for checks that are considered somewhat safe, and for for this we use the
static exchange evaluation which only considers the `to_sq` of a move.
This is not reliable for discovered checks, where another piece is giving
the check and is arguably a more dangerous type of check. Thus, if the check
is a discovered check, the result of SEE is not relevant and can be ignored.
There was a simplification attempt last week for the tropism
term in king danger, which passed STC but failed LTC. This
was an indirect sign that maybe the tropism factor was sightly
untuned in current master, so we tried to change it from 1/4
to 5/16.
Vizvezdenec [Fri, 1 Feb 2019 06:21:23 +0000 (09:21 +0300)]
More precise checks evaluation in king danger
Remove overlapping safe checks from kingdanger:
- rook and queen checks from the same square: rook check is preferred
- bishop and queen checks form the same square: queen check is preferred
Increase bishop and rook check values as a compensation.
I have quite a few ideas of how to improve this patch.
- actually rethinking it now it will maybe be useful to discount
queen/bishop checks if there is only one square that they can
give check from and it's "occupied" by more valuable check. Right
now count of this squares does not really matter.
- maybe some small extra bonus can be given for overlapping checks.
- some ideas about using popcount() on safechecks can be retried.
- tune this safecheck values since they were more or less randomly handcrafted in this patch.
DU-jdto [Wed, 23 Jan 2019 04:19:10 +0000 (15:19 +1100)]
Don't update pvHit after IID
This patch removes line 875 of search.cpp, which was updating pvHit after IID.
Bench testing at depth 22 shows that line 875 of search.cpp never changes the
value of pvHit at NonPV nodes, while at PV nodes it often changes the value
from true to false (and never the reverse). This is because the definition of
pvHit at line 642 is :
while the assignment after IID omits the ` (PvNode && depth > 4 * ONE_PLY) `
condition. As such, unlike the other two post-IID tte reads, this line of code
does not make SF's state more consistent, but rather introduces an inconsistency
in the definition of pvHit. Indeed, changing line 875 read
Miguel Lahoz [Fri, 25 Jan 2019 06:37:03 +0000 (14:37 +0800)]
Change pinning logic in Static Exchange Evaluation (SEE)
This changes 2 parts with regards to static exchange evaluation.
Currently, we do not allow pinned pieces to recapture if *all* opponent
pinners are still in their starting squares. This changes that to having
a less strict requirement, checking if *any* pinners are still in their
starting square. This makes our SEE give more respect to the pinning
side with regards to exchanges, which makes sense because it helps our
search explore more tactical options.
Furthermore, we change the logic for saving pinners into our state
variable when computing slider_blockers. We will include double pinners,
where two sliders may be looking at the same blocker, a similar concept
to our mobility calculation for sliders in our evaluation section.
Interestingly, I think SEE is the only place where the pinners bitboard
is actually used, so as far as I know there are no other side effects
to this change.
An example and some insights:
White Bf2, Kg1
Black Qe3, Bc5
The move Qg3 will be given the correct value of 0. (Previously < 0)
The move Qd4 will be incorrectly given a value of 0. (Previously < 0)
It seems the tradeoff in search is worth it. Qd4 will likely be pruned
soon by something like probcut anyway, while Qg3 could help us spot
tactics at an earlier depth.
stopOnPonderhit is used to stop search quickly on a ponderhit. It is set by mainThread as part of its time management. However, master employs it as a signal between mainThread and the UCI thread. This is not necessary, it is sufficient for the UCI thread to signal that pondering finished, and mainThread should do its usual time-keeping job, and in this case stop immediately.
This patch implements this, removing stopOnPonderHit as an atomic variable from the ThreadPool,
and moving it as a normal variable to mainThread, reducing its scope. In MainThread::check_time() the search is stopped immediately if ponder switches to false, and the variable stopOnPonderHit is set.
Furthermore, ponder has been moved to mainThread, as the variable is only used to exchange signals between the UCI thread and mainThread.
The version has been tested locally (as fishtest doesn't support ponder):
Score of ponderSimp vs master: 2616 - 2528 - 8630 [0.503] 13774
Elo difference: 2.22 +/- 3.54
Jonathan D [Sun, 20 Jan 2019 11:20:21 +0000 (19:20 +0800)]
Tweak initiative and Pawn PSQT (#1957)
Small changes in initiative(). For Pawn PSQT, endgame values for d6-e6 and d7-e7 are now symmetric. The MG value of d2 is now smaller than e2 (d2=13, e2=21 now compared to d2=19, e2=16 before). The MG values of h5-h6-h7 also increased so this might encourage stockfish for more h-pawn pushes.
protonspring [Tue, 15 Jan 2019 05:53:43 +0000 (22:53 -0700)]
Remove AdjacentFiles
This is a non-functional simplification that removes the AdjacentFiles array.
This array is simple enough to calculate that the pre-calculated array provides
no benefit. Reduces the memory footprint.
The new form is likely to trigger a bit more at LTC. Given that LTC
appears to be an improvement, I think that is fine.
The change is not very invasive: it does the same as before, use
potentially less time for moves that are very stable. Most of the
time, the full bonus was given if the bonus was given, so the gradual
part {3, 4, 5} didn't matter much. Whereas previously 'stable' was
expressed as the last 80% of iterations are the same, now I use a
fixed depth (10 iterations). For TCEC style TC, it will presumably
imply some more moves that are played quicker (and thus more time
on the clock when it potentially matters). Note that 10 iterations
of stability means we've been proposing that move for 99.9% of search
time.
The variable pvExact now overlaps with the pvHit concept. So you simplify
the logic with small code tweaks to have pvHit trigger where pvExact
previously triggered.
MJZ1977 [Wed, 9 Jan 2019 14:05:28 +0000 (15:05 +0100)]
Flag critical search tree in hash table
Introducing new concept, saving principal lines into the transposition table
to generate a "critical search tree" which we can reuse later for intelligent
pruning/extension decisions.
For instance in this patch we just reduce reduction for these lines. But a lot
of other ideas are possible.
To go further : tune some parameters, how to add or remove lines from the
critical search tree, how to use these lines in search choices, etc.
Miguel Lahoz [Sat, 5 Jan 2019 20:53:21 +0000 (04:53 +0800)]
Introduce Multi-Cut
This was inspired after reading about
[Multi-Cut](https://www.chessprogramming.org/Multi-Cut).
We now do non-singular cut node pruning. The idea is to prune when we
have a "backup plan" in case our expected fail high node does not fail
high on the ttMove.
For singular extensions, we do a search on all other moves but the
ttMove. If this fails high on our original beta, this means that both
the ttMove, as well as at least one other move was proven to fail high
on a lower depth search. We then assume that one of these moves will
work on a higher depth and prune.
This addresses partially issue #1911 in that it documents in our
Readme the command that users can use to verifying the md5sum of
their downloaded tablebase files.
Additionally, a quick check of the file size (the size of each
tablebase file modulo 64 is 16 as pointed out by @syzygy1) has been
implemented at launch time in Stockfish.
Closes https://github.com/official-stockfish/Stockfish/pull/1927
and https://github.com/official-stockfish/Stockfish/issues/1911
protonspring [Tue, 1 Jan 2019 12:36:56 +0000 (05:36 -0700)]
Remove "Any" predicate filter (#1914)
This custom predicate filter creates an unnecessary abstraction layer, but doesn't make the code any more readable. The code is clear enough without it.
erbsenzaehler [Sat, 29 Dec 2018 10:49:10 +0000 (11:49 +0100)]
Improve the Readme
I tried to improve the Readme because many people in my local
chess club do not understand some of the UCO options properly.
Starting point of this was Cfish's Readme by Ronald de Man,
some internet resources and the Stockfish code itself.
31m059 [Thu, 27 Dec 2018 06:51:43 +0000 (01:51 -0500)]
Always initialize and evaluate king safety
Recent tests by @xoto10, @Vizvezdenec, and myself seemed to hint that Elo could
be gained by expanding the number of cases where king safety is applied. Several
users (@Spliffjiffer, @Vizvezdenec) have anticipated benefits specifically in
evaluation of tactics. It appears that we actually do not need to restrict the
cases in which we initialize and evaluate king safety at all: initializing and
evaluating it in every position appears roughly Elo-neutral at STC and possibly
a substantial Elo gain at LTC.
Any explanation for this scaling is, at this point, conjecture. Assuming it is
not due to chance, my hypothesis is that initialization of king safety in all
positions is a mild slowdown, offset by an Elo gain of evaluating king safety
in all positions. At STC this produces Elo gains and losses that offset each
other, while at longer time control the slowdown is much less important, leaving
only the Elo gain. It probably helps SF to explore king attacks much earlier in
search with high numbers of enemy pieces concentrating but not essentially attacking
king ring.
Thanks to @xoto10 and @Vizvezdenec for helping run my LTC!
noobpwnftw [Sat, 22 Dec 2018 17:05:13 +0000 (01:05 +0800)]
Fix crash in best_group() (#1891)
This pull request fixes a rare crashing bug on Windows inside our NUMA code, first
reported by Dann Corbit in the following forum thread (thanks!):
https://groups.google.com/forum/?fromgroups=#!topic/fishcooking/gA6aoMEuOwg
The fix is to not use structure member beyond known size when iterating through
'SYSTEM_LOGICAL_PROCESSOR_INFORMATION_EX' structure. We note that the Microsoft
API is guaranteed to provide us at least one element upon successful, and no
element in the structure can have a zero size.
The `&& (ss-1)->killers[0] ` conditions are there seemingly to protect
accessing ss-5.
This is unneeded and not so intuitive (as the killer is checked for equality
with currentMove, and that one is non-zero once we're high enough in the stack,
this protects access to ss-5). We can just extend the stack from ss-4 to ss-5,
so we can call update_continuation_histories(ss-1, ..) always in search.
This goes a bit further than #1881 and addresses a comment in #1878.
protonspring [Mon, 17 Dec 2018 17:25:25 +0000 (10:25 -0700)]
Improve endgame KBN vs K (#1877)
Even when playing without endgame table bases, this particular endgame should
be a win 100% of the time when Stockfish is given a KRBK position, assuming
there are enough moves remaining in the FEN to finish the game without hitting
the 50 move rule.
PROBLEM: The issue with master here is that the PushClose difference per square
is 20, however, the difference in squares for the PushToCorners array is usually
less. Thus, the engine prefers to move the kings closer together rather than pushing
the weak king to the correct corner.
What happens is if the weak king is in a safe corner, SF still prefers pushing the
kings together. Occasionally, the strong king traps the weak king in the safe corner.
It takes a while for SF to figure it out, but often draws the game by the 50 move rule
(on shorter time controls).
This patch increases the PushToCorners values to correct this problem. We also added
an assert to catch any overflow problem if anybody would want to increase the array
values again in the future.
It was tested in a couple of matches starting with random KRBK positions and showed
increased winning rates, see https://github.com/official-stockfish/Stockfish/pull/1877
Ubuntu 16.04 can now be used with travis. Updating all the other stuff
when there.
Invoking the lld linker seems to save 5 minutes with clang on linux.
mstembera [Sun, 23 Dec 2018 15:10:07 +0000 (07:10 -0800)]
Use a bit less code to calculate hashfull() (#1830)
* Use a bit less code to calculate hashfull(). Change post increment to preincrement as is more standard
in the rest of stockfish. Scale result so we report 100% instead of 99.9% when completely full.
protonspring [Sat, 8 Dec 2018 17:08:59 +0000 (10:08 -0700)]
Simplify KBNK endgame implementation
We do not need to change the winnerKSq variable, so we can simplify
a little bit the logic of the code by changing only the loserKSq
variable when it is necessary. Also consolidate and clarify comments.
See the pull request thread for a proof that the code is correct:
https://github.com/official-stockfish/Stockfish/pull/1854
mstembera [Tue, 18 Dec 2018 07:50:57 +0000 (08:50 +0100)]
New voting formula for threads
We now use a quadratic formula during the vote for threads
when deciding on which thread to pick a move from.
time control 5+0.05, with 8 threads:
LLR: 2.95 (-2.94,2.94) [0.00,5.00]
Total: 20202 W: 4031 L: 3813 D: 12358
http://tests.stockfishchess.org/tests/view/5c16c8e60ebc5902ba1223e2
time control 20+0.2, with 8 threads:
LLR: 2.96 (-2.94,2.94) [0.00,5.00]
Total: 14330 W: 2290 L: 2115 D: 9925
http://tests.stockfishchess.org/tests/view/5c16efca0ebc5902ba122993
20000 games match at time control 5+0.05, with 31 threads:
ELO: 5.63 +-2.8 (95%) LOS: 100.0%
Total: 20000 W: 3539 L: 3215 D: 13246
http://tests.stockfishchess.org/tests/view/5c16f07a0ebc5902ba122a20
31m059 [Sat, 15 Dec 2018 06:55:25 +0000 (01:55 -0500)]
Use stronglyProtected
~stronglyProtected is quite similar to ~attackedBy[Them][PAWN] & ~attackedBy2[Them],
the only difference appears to be that the former includes squares attacked twice
by both sides. The resulting logic is simpler, and the change appears to be at least
Elo-neutral at both STC and LTC.
Inspired by Nick Pelling's test
http://tests.stockfishchess.org/tests/view/5c144d110ebc5902ba11e4af
and an older test of mine
http://tests.stockfishchess.org/tests/view/5c0402810ebc5902bcee1fc8
is almost never true, since at the root alpha for depth < 5 is -VALUE_INFINITE.
Nevertheless with the new failHigh scheme, this is not guaranteed, and rootDepth > 5,
can still result in a depth < 2 search at the rootNode. If now another thread,
via the hash, writes a new low eval to the rootPos qsearch can be entered.
Rare but not unseen... I assume that some of the crashes in fishtest recently
might be due to this.
projects / stockfish / log