Steinar H. Gunderson [Fri, 16 Jul 2021 11:25:36 +0000 (13:25 +0200)]
Merge remote-tracking branch 'upstream/master' into HEAD
VoyagerOne [Mon, 12 Jul 2021 18:44:29 +0000 (14:44 -0400)]
Don't save excluded move eval in TT
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 17544 W: 1384 L: 1236 D: 14924
Ptnml(0-2): 37, 1031, 6499, 1157, 48
https://tests.stockfishchess.org/tests/view/
60ec8d9bd1189bed71812999
LTC:
LLR: 2.95 (-2.94,2.94) <0.50,3.50>
Total: 26136 W: 823 L: 707 D: 24606
Ptnml(0-2): 6, 643, 11656, 755, 8
https://tests.stockfishchess.org/tests/view/
60ecb11ed1189bed718129ba
closes https://github.com/official-stockfish/Stockfish/pull/3614
Bench:
5505251
Vizvezdenec [Sat, 10 Jul 2021 21:09:15 +0000 (00:09 +0300)]
Remove second futility pruning depth limit
This patch removes futility pruning lmrDepth limit for futility pruning at parent nodes.
Since it's already capped by margin that is a function of lmrDepth there is no need to extra cap it with lmrDepth.
passed STC
https://tests.stockfishchess.org/tests/view/
60e9b5dfd1189bed71812777
LLR: 2.97 (-2.94,2.94) <-2.50,0.50>
Total: 14872 W: 1264 L: 1145 D: 12463
Ptnml(0-2): 37, 942, 5369, 1041, 47
passed LTC
https://tests.stockfishchess.org/tests/view/
60e9c635d1189bed71812790
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 40336 W: 1280 L: 1225 D: 37831
Ptnml(0-2): 24, 1057, 17960, 1094, 33
closes https://github.com/official-stockfish/Stockfish/pull/3612
bench:
5064969
pb00067 [Wed, 7 Jul 2021 12:32:54 +0000 (14:32 +0200)]
SEE: simplify stm variable initialization
Pull #3458 removed the only usage of pos.see_ge() moving pieces that
don't belong to the side to move, so we can simplify this, adding an assert.
closes https://github.com/official-stockfish/Stockfish/pull/3607
No functional change
Vizvezdenec [Tue, 6 Jul 2021 17:44:50 +0000 (20:44 +0300)]
Remove futility pruning depth limit
This patch removes futility pruning depth limit for child node futility pruning.
In current master it was double capped by depth and by futility margin, which is also a function of depth, which didn't make much sense.
passed STC
https://tests.stockfishchess.org/tests/view/
60e2418f9ea99d7c2d693e64
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 116168 W: 9100 L: 9097 D: 97971
Ptnml(0-2): 319, 7496, 42476, 7449, 344
passed LTC
https://tests.stockfishchess.org/tests/view/
60e3374f9ea99d7c2d693f20
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 43304 W: 1282 L: 1231 D: 40791
Ptnml(0-2): 8, 1126, 19335, 1173, 10
closes https://github.com/official-stockfish/Stockfish/pull/3606
bench
4965493
SFisGOD [Fri, 2 Jul 2021 22:13:13 +0000 (06:13 +0800)]
Update default net to nn-
9e3c6298299a.nnue
Optimization of nn-
956480d8378f.nnue using SPSA
https://tests.stockfishchess.org/tests/view/
60da2bf63beab81350ac9fe7
Same method as described in PR #3593
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 17792 W: 1525 L: 1372 D: 14895
Ptnml(0-2): 28, 1156, 6401, 1257, 54
https://tests.stockfishchess.org/tests/view/
60deffc59ea99d7c2d693c19
LTC:
LLR: 2.96 (-2.94,2.94) <0.50,3.50>
Total: 36544 W: 1245 L: 1109 D: 34190
Ptnml(0-2): 12, 988, 16139, 1118, 15
https://tests.stockfishchess.org/tests/view/
60df11339ea99d7c2d693c22
closes https://github.com/official-stockfish/Stockfish/pull/3601
Bench:
4687476
Paul Mulders [Tue, 29 Jun 2021 09:13:54 +0000 (11:13 +0200)]
Allow passing RTLIB=compiler-rt to make
Not all linux users will have libatomic installed.
When using clang as the system compiler with compiler-rt as the default
runtime library instead of libgcc, atomic builtins may be provided by compiler-rt.
This change allows such users to pass RTLIB=compiler-rt to make sure
the build doesn't error out on the missing (unnecessary) libatomic.
closes https://github.com/official-stockfish/Stockfish/pull/3597
No functional change
candirufish [Thu, 1 Jul 2021 17:51:41 +0000 (19:51 +0200)]
no cut node reduction for killer moves.
stc:
LLR: 2.95 (-2.94,2.94) <-0.50,2.50>
Total: 44344 W: 3474 L: 3294 D: 37576
Ptnml(0-2): 117, 2710, 16338, 2890, 117
https://tests.stockfishchess.org/tests/view/
60d8ea673beab81350ac9eb8
ltc:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 82600 W: 2638 L: 2441 D: 77521
Ptnml(0-2): 38, 2147, 36749, 2312, 54
https://tests.stockfishchess.org/tests/view/
60d9048f3beab81350ac9eed
closes https://github.com/official-stockfish/Stockfish/pull/3600
Bench:
5160239
xoto10 [Wed, 30 Jun 2021 08:22:59 +0000 (09:22 +0100)]
Simplify lazy_skip.
Small speedup by removing operations in lazy_skip.
STC 10+0.1 :
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 55088 W: 4553 L: 4482 D: 46053
Ptnml(0-2): 163, 3546, 20045, 3637, 153
https://tests.stockfishchess.org/tests/view/
60daa2cb3beab81350aca04d
LTC 60+0.6 :
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 46136 W: 1457 L: 1407 D: 43272
Ptnml(0-2): 10, 1282, 20442, 1316, 18
https://tests.stockfishchess.org/tests/view/
60db0e753beab81350aca08e
closes https://github.com/official-stockfish/Stockfish/pull/3599
Bench
5122403
Stéphane Nicolet [Wed, 23 Jun 2021 07:55:42 +0000 (09:55 +0200)]
Simplify format_cp_aligned_dot()
closes https://github.com/official-stockfish/Stockfish/pull/3583
No functional change
Joost VandeVondele [Sat, 3 Jul 2021 07:20:06 +0000 (09:20 +0200)]
Restore development version
No functional change
Joost VandeVondele [Mon, 28 Jun 2021 19:46:04 +0000 (21:46 +0200)]
Stockfish 14
Official release version of Stockfish 14
Bench:
4770936
---
Today, we have the pleasure to announce Stockfish 14.
As usual, downloads will be freely available at https://stockfishchess.org
The engine is now significantly stronger than just a few months ago,
and wins four times more game pairs than it loses against the previous
release version [0]. Stockfish 14 is now at least 400 Elo ahead of
Stockfish 7, a top engine in 2016 [1]. During the last five years,
Stockfish has thus gained about 80 Elo per year.
Stockfish 14 evaluates positions more accurately than Stockfish 13 as
a result of two major steps forward in defining and training the
efficiently updatable neural network (NNUE) that provides the evaluation
for positions.
First, the collaboration with the Leela Chess Zero team - announced
previously [2] - has come to fruition. The LCZero team has provided a
collection of billions of positions evaluated by Leela that we have
combined with billions of positions evaluated by Stockfish to train the
NNUE net that powers Stockfish 14. The fact that we could use and combine
these datasets freely was essential for the progress made and demonstrates
the power of open source and open data [3].
Second, the architecture of the NNUE network was significantly updated:
the new network is not only larger, but more importantly, it deals better
with large material imbalances and can specialize for multiple phases of
the game [4]. A new project, kick-started by Gary Linscott and
Tomasz Sobczyk, led to a GPU accelerated net trainer written in
pytorch.[5] This tool allows for training high-quality nets in a couple
of hours.
Finally, this release features some search refinements, minor bug
fixes and additional improvements. For example, Stockfish is now about
90 Elo stronger for chess960 (Fischer random chess) at short time control.
The Stockfish project builds on a thriving community of enthusiasts
(thanks everybody!) that contribute their expertise, time, and resources
to build a free and open-source chess engine that is robust, widely
available, and very strong. We invite our chess fans to join the fishtest
testing framework and programmers to contribute to the project on
github [6].
Stay safe and enjoy chess!
The Stockfish team
[0] https://tests.stockfishchess.org/tests/view/
60dae5363beab81350aca077
[1] https://nextchessmove.com/dev-builds
[2] https://stockfishchess.org/blog/2021/stockfish-13/
[3] https://lczero.org/blog/2021/06/the-importance-of-open-data/
[4] https://github.com/official-stockfish/Stockfish/commit/
e8d64af1
[5] https://github.com/glinscott/nnue-pytorch/
[6] https://stockfishchess.org/get-involved/
Brad Knox [Tue, 29 Jun 2021 06:40:16 +0000 (01:40 -0500)]
Update Top CPU Contributors
closes https://github.com/official-stockfish/Stockfish/pull/3595
No functional change
SFisGOD [Mon, 28 Jun 2021 06:58:51 +0000 (14:58 +0800)]
Update default net to nn-
3475407dc199.nnue
Optimization of eight subnetwork output layers of Michael's nn-
190f102a22c3.nnue using SPSA
https://tests.stockfishchess.org/tests/view/
60d5510642a522cc50282ef3
Parameters: A total of 256 net weights and 8 net biases were tuned
New best values: The raw values at the end of the tuning run were used (800k games, 5 seconds TC)
Settings: default ck value and SPSA A is 30,000 (3.75% of the total number of games)
STC:
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 29064 W: 2435 L: 2269 D: 24360
Ptnml(0-2): 72, 1857, 10505, 2029, 69
https://tests.stockfishchess.org/tests/view/
60d8ea123beab81350ac9eb6
LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 61848 W: 2055 L: 1884 D: 57909
Ptnml(0-2): 18, 1708, 27310, 1861, 27
https://tests.stockfishchess.org/tests/view/
60d8f0393beab81350ac9ec6
closes https://github.com/official-stockfish/Stockfish/pull/3593
Bench:
4770936
MichaelB7 [Sun, 27 Jun 2021 15:26:09 +0000 (11:26 -0400)]
Make net nn-
956480d8378f.nnue the default
Trained with the pytorch trainer: https://github.com/glinscott/nnue-pytorch
python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_18 --resume-from-model ./pt/nn-
75980ca503c6.pt
This run is thus started from a previous master net.
all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
Each set was concatenated together - making one large Wrong_NNUE 2 binpack and one large Training so the were approximately equal in size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training_Data binpack
passed STC:
https://tests.stockfishchess.org/tests/view/
60d0c0a7a8ec07dc34c072b2
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 18440 W: 1693 L: 1531 D: 15216
Ptnml(0-2): 67, 1225, 6464, 1407, 57
passed LTC:
https://tests.stockfishchess.org/tests/view/
60d762793beab81350ac9d72
LLR: 2.98 (-2.94,2.94) <0.50,3.50>
Total: 93120 W: 3152 L: 2933 D: 87035
Ptnml(0-2): 48, 2581, 41076, 2814, 41
passed LTC (rebased branch to current master):
https://tests.stockfishchess.org/tests/view/
60d85eeb3beab81350ac9e2b
LLR: 2.96 (-2.94,2.94) <0.50,3.50>
Total: 42688 W: 1347 L: 1206 D: 40135
Ptnml(0-2): 14, 1097, 18981, 1238, 14.
closes https://github.com/official-stockfish/Stockfish/pull/3592
Bench:
4906727
Joost VandeVondele [Wed, 23 Jun 2021 05:23:21 +0000 (07:23 +0200)]
Update WDL model for NNUE
This updates the WDL model based on the LTC statistics in June this year (10M games),
so from pre-NNUE to NNUE based results.
(for old results see, https://github.com/official-stockfish/Stockfish/pull/2778)
As before the fit by the model to the data is quite good.
closes https://github.com/official-stockfish/Stockfish/pull/3582
No functional change
bmc4 [Tue, 22 Jun 2021 22:33:14 +0000 (19:33 -0300)]
Simplify Reductions Initialization
passed
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 45032 W: 3600 L: 3518 D: 37914
Ptnml(0-2): 111, 2893, 16435, 2957, 120
https://tests.stockfishchess.org/tests/view/
60d2655d40925195e7a6c527
LTC:
LLR: 3.00 (-2.94,2.94) <-2.50,0.50>
Total: 25728 W: 786 L: 722 D: 24220
Ptnml(0-2): 5, 650, 11494, 706, 9
https://tests.stockfishchess.org/tests/view/
60d2b14240925195e7a6c577
closes https://github.com/official-stockfish/Stockfish/pull/3584
bench:
4602977
Stéphane Nicolet [Tue, 22 Jun 2021 07:08:37 +0000 (09:08 +0200)]
Detect fortresses a little bit quicker
In the so-called "hybrid" method of evaluation of current master, we use the
classical eval (because of its speed) instead of the NNUE eval when the classical
material balance approximation hints that the position is "winning enough" to
rely on the classical eval.
This trade-off idea between speed and accuracy works well in general, but in
some fortress positions the classical eval is just bad. So in shuffling branches
of the search tree, we (slowly) increase the thresehold so that eventually we
don't trust classical anymore and switch to NNUE evaluation.
This patch increases that threshold faster, so that we switch to NNUE quicker
in shuffling branches. Idea is to incite Stockfish to spend less time in fortresses
lines in the search tree, and spend more time searching the critical lines.
passed STC:
LLR: 2.96 (-2.94,2.94) <-0.50,2.50>
Total: 47872 W: 3908 L: 3720 D: 40244
Ptnml(0-2): 122, 3053, 17419, 3199, 143
https://tests.stockfishchess.org/tests/view/
60cef34b457376eb8bcab79d
passed LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 73616 W: 2326 L: 2143 D: 69147
Ptnml(0-2): 21, 1940, 32705, 2119, 23
https://tests.stockfishchess.org/tests/view/
60cf6d842114332881e73528
Retested at LTC against lastest master:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 18264 W: 642 L: 532 D: 17090
Ptnml(0-2): 6, 479, 8055, 583, 9
https://tests.stockfishchess.org/tests/view/
60d18cd540925195e7a6c351
closes https://github.com/official-stockfish/Stockfish/pull/3578
Bench:
5139233
MichaelB7 [Mon, 21 Jun 2021 12:10:35 +0000 (08:10 -0400)]
Make net nn-
190f102a22c3.nnue the default net.
Trained with the pytorch trainer: https://github.com/glinscott/nnue-pytorch
python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_17 --resume-from-model ./pt/nn-
75980ca503c6.pt
This run is thus started from the previous master net.
all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
Each set was concatenated together - making one large Wrong_NNUE 2 binpack and one large Training so the were approximately equal in size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training_Data binpack
passed LTC
https://tests.stockfishchess.org/tests/view/
60d09f52b4c17000d679517f
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 32184 W: 1100 L: 970 D: 30114
Ptnml(0-2): 10, 878, 14193, 994, 17
passed STC
https://tests.stockfishchess.org/tests/view/
60d086c02114332881e7368e
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 11360 W: 1056 L: 906 D: 9398
Ptnml(0-2): 25, 735, 4026, 853, 41
closes https://github.com/official-stockfish/Stockfish/pull/3576
Bench:
4631244
Joost VandeVondele [Mon, 21 Jun 2021 06:23:50 +0000 (08:23 +0200)]
Fix build error on OSX
directly use integer version for cp calculation.
fixes https://github.com/official-stockfish/Stockfish/issues/3573
closes https://github.com/official-stockfish/Stockfish/pull/3574
No functional change
Stéphane Nicolet [Wed, 16 Jun 2021 05:23:26 +0000 (07:23 +0200)]
Remove the Contempt UCI option
This patch removes the UCI option for setting Contempt in classical evaluation.
It is exactly equivalent to using Contempt=0 for the UCI contempt value and keeping
the dynamic part in the algo (renaming this dynamic part `trend` to better describe
what it does). We have tried quite hard to implement a working Contempt feature for
NNUE but nothing really worked, so it is probably time to give up.
Interested chess fans wishing to keep playing with the UCI option for Contempt and
use it with the classical eval are urged to download the version tagged "SF_Classical"
of Stockfish (dated 31 July 2020), as it was the last version where our search
algorithm was tuned for the classical eval and is probably our strongest classical
player ever: https://github.com/official-stockfish/Stockfish/tags
Passed STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 72904 W: 6228 L: 6175 D: 60501
Ptnml(0-2): 221, 5006, 25971, 5007, 247
https://tests.stockfishchess.org/tests/view/
60c98bf9457376eb8bcab18d
Passed LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 45168 W: 1601 L: 1547 D: 42020
Ptnml(0-2): 38, 1331, 19786, 1397, 32
https://tests.stockfishchess.org/tests/view/
60c9c7fa457376eb8bcab1bb
closes https://github.com/official-stockfish/Stockfish/pull/3575
Bench:
4947716
Stéphane Nicolet [Sun, 20 Jun 2021 08:29:20 +0000 (10:29 +0200)]
Keep more pawns and pieces when attacking
This patch increase the weight of pawns and pieces from 28 to 32
in the scaling formula we apply to the output of the NNUE pure eval.
Increasing this gradient for pawns and pieces means that Stockfish
will try a little harder to keep material when she has the advantage,
and try a little bit harder to escape into an endgame when she is
under pressure.
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 53168 W: 4371 L: 4177 D: 44620
Ptnml(0-2): 160, 3389, 19283, 3601, 151
https://tests.stockfishchess.org/tests/view/
60cefd1d457376eb8bcab7ab
LTC:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 10888 W: 386 L: 288 D: 10214
Ptnml(0-2): 3, 260, 4821, 356, 4
https://tests.stockfishchess.org/tests/view/
60cf709d2114332881e7352b
closes https://github.com/official-stockfish/Stockfish/pull/3571
Bench:
4965430
MichaelB7 [Sat, 19 Jun 2021 13:57:09 +0000 (09:57 -0400)]
Make net nn-
75980ca503c6.nnue the default.
trained with the Python command
c:\nnue>python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_10 --resume-from-model ./pt/nn-
3b20abec10c1.pt
`
all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
Each set was concatenated together - making one large Wrong_NNUE 2 binpack and one large Training so the were approximately equal in size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training_Data binpack .
Net nn-
3b20abec10c1.nnue was chosen as the --resume-from-model with the idea that through learning, the manually hex edited values will be learned and will not need to be manually adjusted going forward. They would also be fine tuned by the learning process.
passed STC:
https://tests.stockfishchess.org/tests/view/
60cdf91e457376eb8bcab66f
LLR: 2.95 (-2.94,2.94) <-0.50,2.50>
Total: 18256 W: 1639 L: 1479 D: 15138
Ptnml(0-2): 59, 1179, 6505, 1313, 72
passed LTC:
https://tests.stockfishchess.org/tests/view/
60ce2166457376eb8bcab6e1
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 18792 W: 654 L: 542 D: 17596
Ptnml(0-2): 9, 490, 8291, 592, 14
closes https://github.com/official-stockfish/Stockfish/pull/3570
Bench:
5020972
Tomasz Sobczyk [Thu, 17 Jun 2021 10:36:06 +0000 (12:36 +0200)]
Change trace with NNUE eval support
This patch adds some more output to the `eval` command. It adds a board display
with estimated piece values (method is remove-piece, evaluate, put-piece), and
splits the NNUE evaluation with (psqt,layers) for each bucket for the NNUE net.
Example:
```
./stockfish
position fen 3Qb1k1/1r2ppb1/pN1n2q1/Pp1Pp1Pr/4P2p/4BP2/4B1R1/1R5K b - - 11 40
eval
Contributing terms for the classical eval:
+------------+-------------+-------------+-------------+
| Term | White | Black | Total |
| | MG EG | MG EG | MG EG |
+------------+-------------+-------------+-------------+
| Material | ---- ---- | ---- ---- | -0.73 -1.55 |
| Imbalance | ---- ---- | ---- ---- | -0.21 -0.17 |
| Pawns | 0.35 -0.00 | 0.19 -0.26 | 0.16 0.25 |
| Knights | 0.04 -0.08 | 0.12 -0.01 | -0.08 -0.07 |
| Bishops | -0.34 -0.87 | -0.17 -0.61 | -0.17 -0.26 |
| Rooks | 0.12 0.00 | 0.08 0.00 | 0.04 0.00 |
| Queens | 0.00 0.00 | -0.27 -0.07 | 0.27 0.07 |
| Mobility | 0.84 1.76 | 0.01 0.66 | 0.83 1.10 |
|King safety | -0.99 -0.17 | -0.72 -0.10 | -0.27 -0.07 |
| Threats | 0.27 0.27 | 0.73 0.86 | -0.46 -0.59 |
| Passed | 0.00 0.00 | 0.79 0.82 | -0.79 -0.82 |
| Space | 0.61 0.00 | 0.24 0.00 | 0.37 0.00 |
| Winnable | ---- ---- | ---- ---- | 0.00 -0.03 |
+------------+-------------+-------------+-------------+
| Total | ---- ---- | ---- ---- | -1.03 -2.14 |
+------------+-------------+-------------+-------------+
NNUE derived piece values:
+-------+-------+-------+-------+-------+-------+-------+-------+
| | | | Q | b | | k | |
| | | | +12.4 | -1.62 | | | |
+-------+-------+-------+-------+-------+-------+-------+-------+
| | r | | | p | p | b | |
| | -3.89 | | | -0.84 | -1.19 | -3.32 | |
+-------+-------+-------+-------+-------+-------+-------+-------+
| p | N | | n | | | q | |
| -1.81 | +3.71 | | -4.82 | | | -5.04 | |
+-------+-------+-------+-------+-------+-------+-------+-------+
| P | p | | P | p | | P | r |
| +1.16 | -0.91 | | +0.55 | +0.12 | | +0.50 | -4.02 |
+-------+-------+-------+-------+-------+-------+-------+-------+
| | | | | P | | | p |
| | | | | +2.33 | | | +1.17 |
+-------+-------+-------+-------+-------+-------+-------+-------+
| | | | | B | P | | |
| | | | | +4.79 | +1.54 | | |
+-------+-------+-------+-------+-------+-------+-------+-------+
| | | | | B | | R | |
| | | | | +4.54 | | +6.03 | |
+-------+-------+-------+-------+-------+-------+-------+-------+
| | R | | | | | | K |
| | +4.81 | | | | | | |
+-------+-------+-------+-------+-------+-------+-------+-------+
NNUE network contributions (Black to move)
+------------+------------+------------+------------+
| Bucket | Material | Positional | Total |
| | (PSQT) | (Layers) | |
+------------+------------+------------+------------+
| 0 | + 0.32 | - 1.46 | - 1.13 |
| 1 | + 0.25 | - 0.68 | - 0.43 |
| 2 | + 0.46 | - 1.72 | - 1.25 |
| 3 | + 0.55 | - 1.80 | - 1.25 |
| 4 | + 0.48 | - 1.77 | - 1.29 |
| 5 | + 0.40 | - 2.00 | - 1.60 |
| 6 | + 0.57 | - 2.12 | - 1.54 | <-- this bucket is used
| 7 | + 3.38 | - 2.00 | + 1.37 |
+------------+------------+------------+------------+
Classical evaluation -1.00 (white side)
NNUE evaluation +1.54 (white side)
Final evaluation +2.38 (white side) [with scaled NNUE, hybrid, ...]
```
Also renames the export_net() function to save_eval() while there.
closes https://github.com/official-stockfish/Stockfish/pull/3562
No functional change
proukornew [Fri, 18 Jun 2021 21:52:46 +0000 (00:52 +0300)]
Fix for Cygwin's environment build-profile (fixed)
The Cygwin environment has two g++ compilers, each with a different problem
for compiling Stockfish at the moment:
(a) g++.exe : full posix build compiler, linked to cygwin dll.
=> This one has a problem embedding the net.
(b) x86_64-w64-mingw32-g++.exe : native Windows build compiler.
=> This one manages to embed the net, but has a problem related to libgcov
when we use the profile-build target of Stockfish.
This patch solves the problem for compiler (b), so that our recommended command line
if you want to build an optimized version of Stockfish on Cygwin becomes something
like the following (you can change the ARCH value to whatever you want, but note
the COMP and CXX variables pointing at the right compiler):
```
make -j profile-build ARCH=x86-64-modern COMP=mingw CXX=x86_64-w64-mingw32-c++.exe
```
closes https://github.com/official-stockfish/Stockfish/pull/3569
No functional change
Joost VandeVondele [Fri, 18 Jun 2021 21:50:01 +0000 (23:50 +0200)]
Make net nn-
50144f835024.nnue the default
trained with the Python command
c:\nnue>python train.py i:/bin/all.binpack i:/bin/all.binpack --gpus 1 --threads 4 --num-workers 30 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed %random%%random% --default_root_dir exp/run_8 --resume-from-model ./pt/nn-
6ad41a9207d0.pt
`
all.binpack equaled 4 parts Wrong_NNUE_2.binpack https://drive.google.com/file/d/1seGNOqcVdvK_vPNq98j-zV3XPE5zWAeq/view?usp=sharing plus two parts of Training_Data.binpack https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
Each set was concatenated together - make one large Wrong_NNUE 2 binpack and one large Training_Data of approximate size. They were then interleaved together. The idea was to give Wrong_NNUE.binpack closer to equal weighting with the Training _Data binpack .
nn-
6ad41a9207d0.pt was derived from a net vondele ran which passed STC quickly,
but faltered in LTC. https://tests.stockfishchess.org/tests/view/
60cba666457376eb8bcab443
STC:
LLR: 2.95 (-2.94,2.94) <-0.50,2.50>
Total: 18792 W: 2068 L: 1889 D: 14835
Ptnml(0-2): 82, 1480, 6117, 1611, 106
https://tests.stockfishchess.org/tests/view/
60ccda8b457376eb8bcab568
LTC:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 11376 W: 574 L: 454 D: 10348
Ptnml(0-2): 4, 412, 4747, 510, 15
https://tests.stockfishchess.org/tests/view/
60ccf952457376eb8bcab58d
closes https://github.com/official-stockfish/Stockfish/pull/3568
Bench:
4900906
Tomasz Sobczyk [Fri, 18 Jun 2021 10:03:03 +0000 (12:03 +0200)]
Add basic github workflow
move to github actions to replace travis CI.
First version, testing on linux using gcc and clang.
gcc build with sanitizers and valgrind.
No functional change
SFisGOD [Fri, 18 Jun 2021 19:09:20 +0000 (03:09 +0800)]
Update default net to nn-
aa9d7eeb397e.nnue
Optimization of vondele's nn-
33c9d39e5eb6.nnue using SPSA
https://tests.stockfishchess.org/tests/view/
60ca68be457376eb8bcab28b
Setting: ck values are default based on how large the parameters are
The new values for this net are the raw values at the end of the tuning (80k games)
The significant changes are in buckets 1 and 2 (5-12 pieces) so the main difference is in playing endgames if we compare it to nn-33c9. There is also change in bucket 7 (29-32 pieces) but not as substantial as the changes in buckets 1 and 2. If we interpret the changes based on an experiment a few months ago, this new net plays more optimistically during endgames and less optimistically during openings.
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 49504 W: 4246 L: 4053 D: 41205
Ptnml(0-2): 140, 3282, 17749, 3407, 174
https://tests.stockfishchess.org/tests/view/
60cbd752457376eb8bcab478
LTC:
LLR: 2.95 (-2.94,2.94) <0.50,3.50>
Total: 88720 W: 4926 L: 4651 D: 79143
Ptnml(0-2): 105, 4048, 35793, 4295, 119
https://tests.stockfishchess.org/tests/view/
60cc7828457376eb8bcab4fa
closes https://github.com/official-stockfish/Stockfish/pull/3566
Bench:
4758885
ap [Thu, 17 Jun 2021 23:43:58 +0000 (01:43 +0200)]
New default net nn-
3b20abec10c1.nnue
This net was created by @pleomati, who manually edited with an hex editor
10 values randomly chosen in the LCSFNet10 net (nn-
6ad41a9207d0.nnue) to
create this one. The LCSFNet10 net was trained by Joost VandeVondele from
a dataset combining Stockfish games and Leela games (16x10^9 positions from
SF self-play at depth 9, and 6.3x10^9 positions from Leela games, so overall
72% of Stockfish positions and 28% of Leela positions).
passed STC 10+0.1:
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 50888 W: 5881 L: 5654 D: 39353
Ptnml(0-2): 281, 4290, 16085, 4497, 291
https://tests.stockfishchess.org/tests/view/
60cbfa68457376eb8bcab49a
passed LTC 60+0.6:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 25480 W: 1498 L: 1338 D: 22644
Ptnml(0-2): 36, 1155, 10193, 1325, 31
https://tests.stockfishchess.org/tests/view/
60cc4af8457376eb8bcab4d4
closes https://github.com/official-stockfish/Stockfish/pull/3564
Bench:
4904930
Stéphane Nicolet [Thu, 17 Jun 2021 16:09:42 +0000 (18:09 +0200)]
Revert "Fix for Cygwin's environment build-profile"
This reverts commit "Fix for Cygwin's environment build-profile", as it was
giving errors for "make clean" on some Windows environments. See comments in
https://github.com/official-stockfish/Stockfish/commit/
68bf362ea2385a641be9f5ed9ce2acdf55a1ecf1
Possibly somebody can propose a solution that would fix Cygwin builds and
not break on other system too, stay tuned! :-)
No functional change
bmc4 [Tue, 15 Jun 2021 23:56:09 +0000 (20:56 -0300)]
Simplify reduction when best move doesn't change frequently.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 40400 W: 3468 L: 3377 D: 33555
Ptnml(0-2): 134, 2734, 14388, 2795, 149
https://tests.stockfishchess.org/tests/view/
60c93e5a457376eb8bcab15f
LTC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 34200 W: 1190 L: 1128 D: 31882
Ptnml(0-2): 22, 998, 15001, 1054, 25
https://tests.stockfishchess.org/tests/view/
60c96a1a457376eb8bcab180
closes https://github.com/official-stockfish/Stockfish/pull/3559
bench:
5629669
proukornew [Thu, 13 May 2021 21:49:28 +0000 (00:49 +0300)]
Fix for Cygwin's environment build-profile
The Cygwin environment has two g++ compilers, each with a different problem
for compiling Stockfish at the moment:
(a) g++.exe : full posix build compiler, linked to cygwin dll.
=> This one has a problem embedding the net.
(b) x86_64-w64-mingw32-g++.exe : native Windows build compiler.
=> This one manages to embed the net, but has a problem related to libgcov
when we use the profile-build target of Stockfish.
This patch solves the problem for compiler (b), so that our recommended command line
if you want to build an optimized version of Stockfish on Cygwin becomes something
like the following (you can change the ARCH value to whatever you want, but note
the COMP and CXX variables pointing at the right compiler):
```
make -j profile-build ARCH=x86-64-modern COMP=mingw CXX=x86_64-w64-mingw32-c++.exe
```
closes https://github.com/official-stockfish/Stockfish/pull/3463
No functional change
Joost VandeVondele [Tue, 15 Jun 2021 10:49:23 +0000 (12:49 +0200)]
New default net nn-
33c9d39e5eb6.nnue
As the previous net, this net is trained on Leela games as provided by borg.
See also https://lczero.org/blog/2021/06/the-importance-of-open-data/
The particular data set, which is a mix of T60 and T74 data, is now available as a single binpack:
https://drive.google.com/file/d/1RFkQES3DpsiJqsOtUshENtzPfFgUmEff/view?usp=sharing
The training command was:
python train.py ../../training_data_pylon.binpack ../../training_data_pylon.binpack --gpus 1 --threads 2 --num-workers 2 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 10 --features=HalfKAv2^ --lambda=1.0 --max_epochs=440 --seed $RANDOM --default_root_dir exp/run_2
passed STC:
https://tests.stockfishchess.org/tests/view/
60c887cb457376eb8bcab054
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 12792 W: 1483 L: 1311 D: 9998
Ptnml(0-2): 62, 989, 4131, 1143, 71
passed LTC:
https://tests.stockfishchess.org/tests/view/
60c8e5c4457376eb8bcab0f0
LLR: 2.95 (-2.94,2.94) <0.50,3.50>
Total: 11272 W: 601 L: 477 D: 10194
Ptnml(0-2): 9, 421, 4657, 535, 14
also had strong LTC performance against another strong net of the series:
https://tests.stockfishchess.org/tests/view/
60c8c40d457376eb8bcab0c6
closes https://github.com/official-stockfish/Stockfish/pull/3557
Bench:
5032320
J. Oster [Mon, 14 Jun 2021 15:28:30 +0000 (17:28 +0200)]
Fix a rare case of wrong TB ranking
of a root move leading to a 3-fold repetition.
With this small fix a draw ranking and thus a draw score is being applied.
This works for both, ranking by dtz or wdl tables.
Fixes https://github.com/official-stockfish/Stockfish/issues/3542
(No functional change without TBs.)
Bench:
4877339
Tomasz Sobczyk [Sat, 12 Jun 2021 18:45:14 +0000 (20:45 +0200)]
Reduce the number of accumulator states
Reduce from 3 to 2. Make the intent of the states clearer.
STC: https://tests.stockfishchess.org/tests/view/
60c50111457376eb8bcaad03
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 61888 W: 5007 L: 4944 D: 51937
Ptnml(0-2): 164, 3947, 22649, 4030, 154
LTC: https://tests.stockfishchess.org/tests/view/
60c52b1c457376eb8bcaad2c
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 20248 W: 688 L: 618 D: 18942
Ptnml(0-2): 7, 551, 8946, 605, 15
closes https://github.com/official-stockfish/Stockfish/pull/3548
No functional change.
JWmer [Sun, 13 Jun 2021 21:48:32 +0000 (23:48 +0200)]
Update default net to nn-
8e47cf062333.nnue
This net is the result of training on data used by the Leela project. More precisely,
we shuffled T60 and T74 data kindly provided by borg (for different Tnn, the data is
a result of Leela selfplay with differently sized Leela nets).
The data is available at vondele's google drive:
https://drive.google.com/drive/folders/1mftuzYdl9o6tBaceR3d_VBQIrgKJsFpl.
The Leela data comes in small chunks of .binpack files. To shuffle them, we simply
used a small python script to randomly rename the files, and then concatenated them
using `cat`. As validation data we picked a file of T60 data. We will further investigate
T74 data.
The training for the NNUE architecture used 200 epochs with the Python trainer from
the Stockfish project. Unlike the previous run we tried with this data, this run does
not have adjusted scaling — not because we didn't want to, but because we forgot.
However, this training randomly skips 40% more positions than previous run. The loss
was very spiky and decreased slower than it does usually.
Training loss: https://github.com/official-stockfish/images/blob/main/training-loss-
8e47cf062333.png
Validation loss: https://github.com/official-stockfish/images/blob/main/validation-loss-
8e47cf062333.png
This is the exact training command:
python train.py --smart-fen-skipping --random-fen-skipping 14 --batch-size 16384 --threads 4 --num-workers 4 --gpus 1 trainingdata\training_data.binpack validationdata\val.binpack
---
10k STC result:
ELO: 3.61 +-3.3 (95%) LOS: 98.4%
Total: 10000 W: 1241 L: 1137 D: 7622
Ptnml(0-2): 68, 841, 3086, 929, 76
https://tests.stockfishchess.org/tests/view/
60c67e50457376eb8bcaae70
10k LTC result:
ELO: 2.71 +-2.4 (95%) LOS: 98.8%
Total: 10000 W: 659 L: 581 D: 8760
Ptnml(0-2): 22, 485, 3900, 579, 14
https://tests.stockfishchess.org/tests/view/
60c69deb457376eb8bcaae98
Passed LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 9648 W: 685 L: 545 D: 8418
Ptnml(0-2): 22, 448, 3740, 596, 18
https://tests.stockfishchess.org/tests/view/
60c6d41c457376eb8bcaaecf
---
closes https://github.com/official-stockfish/Stockfish/pull/3550
Bench:
4877339
Tomasz Sobczyk [Thu, 10 Jun 2021 15:43:42 +0000 (17:43 +0200)]
Register count for feature transformer
Compute optimal register count for feature transformer accumulation dynamically.
This also introduces a change where AVX512 would only use 8 registers instead of 16
(now possible due to a 2x increase in feature transformer size).
closes https://github.com/official-stockfish/Stockfish/pull/3543
No functional change
Vizvezdenec [Sat, 29 May 2021 03:39:14 +0000 (06:39 +0300)]
Do less LMR extensions
This patch restricts LMR extensions (of non-transposition table moves) from being
used when the transposition table move was extended by two plies via singular
extension. This may serve to limit search explosions in certain positions.
This makes a lot of sense because the precondition for the tt-move to have been
singular extended by two plies is that the result of the alternate search (with
excluded the tt-move) has been a hard fail low: it is natural to later search less
for non tt-moves in this situation.
The current state of depth/extensions/reductions management is getting quite tricky
in our search algo, see https://github.com/official-stockfish/Stockfish/pull/3546#issuecomment-
860174549
for some discussion. Suggestions welcome!
Passed STC
https://tests.stockfishchess.org/tests/view/
60c3f293457376eb8bcaac8d
LLR: 2.95 (-2.94,2.94) <-0.50,2.50>
Total: 117984 W: 9698 L: 9430 D: 98856
Ptnml(0-2): 315, 7708, 42703, 7926, 340
passed LTC
https://tests.stockfishchess.org/tests/view/
60c46ea5457376eb8bcaacc7
LLR: 2.97 (-2.94,2.94) <0.50,3.50>
Total: 11280 W: 401 L: 302 D: 10577
Ptnml(0-2): 2, 271, 4998, 364, 5
closes https://github.com/official-stockfish/Stockfish/pull/3546
Bench:
4709974
Stéphane Nicolet [Sun, 13 Jun 2021 07:59:34 +0000 (09:59 +0200)]
Clarify use of UCI options
Update README.md to clarify use of UCI options
closes https://github.com/official-stockfish/Stockfish/pull/3540
No functional change
Tomasz Sobczyk [Wed, 9 Jun 2021 09:21:55 +0000 (11:21 +0200)]
Read NNUE net faster
Load feature transformer weights in bulk on little-endian machines.
This is in particular useful to test new nets with c-chess-cli,
see https://github.com/lucasart/c-chess-cli/issues/44
```
$ time ./stockfish.exe uci
Before : 0m0.914s
After : 0m0.483s
```
No functional change
Joost VandeVondele [Wed, 9 Jun 2021 21:23:13 +0000 (23:23 +0200)]
Limit double extensions
Double extensions can lead to search explosions, for specific positions.
Currently, however, these double extensions are worth about 10Elo and cannot
be removed. This patch instead limits the number of double extensions given
to a maximum of 3.
This fixes https://github.com/official-stockfish/Stockfish/issues/3532
where the following testcase was shown to be problematic:
```
uci
setoption name Hash value 4
setoption name Contempt value 0
ucinewgame
position fen 8/Pk6/8/1p6/8/P1K5/8/6B1 w - - 37 130
go depth 20
```
passed STC:
https://tests.stockfishchess.org/tests/view/
60c13161457376eb8bcaaa0f
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 73256 W: 6114 L: 6062 D: 61080
Ptnml(0-2): 222, 4912, 26306, 4968, 220
passed LTC:
https://tests.stockfishchess.org/tests/view/
60c196fb457376eb8bcaaa6b
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 166440 W: 5559 L: 5594 D: 155287
Ptnml(0-2): 106, 4921, 73197, 4894, 102
closes https://github.com/official-stockfish/Stockfish/pull/3544
Bench:
5067605
bmc4 [Mon, 7 Jun 2021 18:47:37 +0000 (15:47 -0300)]
Simplify promotion move generator
This patch removes Knight promotion checks from Captures. As a consequence,
it also removes this underpromotion from qsearch.
STC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 37776 W: 3113 L: 3023 D: 31640
Ptnml(0-2): 103, 2419, 13755, 2507, 104
https://tests.stockfishchess.org/tests/view/
60be6a06457376eb8bcaa775
LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 39760 W: 1257 L: 1203 D: 37300
Ptnml(0-2): 11, 1079, 17646, 1133, 11
https://tests.stockfishchess.org/tests/view/
60beb972457376eb8bcaa7c5
closes https://github.com/official-stockfish/Stockfish/pull/3536
Bench:
5530620
bmc4 [Sun, 6 Jun 2021 16:31:57 +0000 (13:31 -0300)]
Reduce in LMR reduction on PvNode
reduce reduction in LMR by 1 on PvNode.
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 266080 W: 22438 L: 21996 D: 221646
Ptnml(0-2): 774, 17874, 95376, 18168, 848
https://tests.stockfishchess.org/tests/view/
60bc0661457376eb8bcaa4bb
LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 20144 W: 698 L: 587 D: 18859
Ptnml(0-2): 2, 529, 8906, 626, 9
https://tests.stockfishchess.org/tests/view/
60bcc3f2457376eb8bcaa58d
closes https://github.com/official-stockfish/Stockfish/pull/3534
bench:
5173012
Guy Vreuls [Thu, 3 Jun 2021 14:46:05 +0000 (16:46 +0200)]
Makefile: Extend sanitize support
Enable compiling with multiple sanitizers at once.
Syntax:
make build ARCH=x86-64-avx512 debug=on sanitize="address undefined"
closes https://github.com/official-stockfish/Stockfish/pull/3524
No functional change.
Joost VandeVondele [Thu, 3 Jun 2021 17:18:24 +0000 (19:18 +0200)]
Enhance CI to error on leaks
Add flags to valgrind in our Continuous Integration scripts,
to error on memory leaks.
closes https://github.com/official-stockfish/Stockfish/pull/3525
No functional change.
Guy Vreuls [Fri, 4 Jun 2021 10:20:27 +0000 (12:20 +0200)]
Revert "Simplify En Passant"
This reverts commit
9f8058bd26df1c3ca37b85f811026f1eb82e6524.
Fixes the memory leak discussed in pull request #3523
https://github.com/official-stockfish/Stockfish/pull/3523
Passed non-regression STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 76184 W: 6330 L: 6282 D: 63572
Ptnml(0-2): 202, 5047, 27564, 5059, 220
https://tests.stockfishchess.org/tests/view/
60ba146c457376eb8bcaa2e2
closes https://github.com/official-stockfish/Stockfish/pull/3527
Benched to verify there is no functional change.
Bench:
4364128
Stéphane Nicolet [Fri, 4 Jun 2021 11:56:40 +0000 (13:56 +0200)]
Clean SIMD code a bit
Cleaner vector code structure in feature transformer. This patch just
regroups the parts of the inner loop for each SIMD instruction set.
Tested for non-regression:
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 115760 W: 9835 L: 9831 D: 96094
Ptnml(0-2): 326, 7776, 41715, 7694, 369
https://tests.stockfishchess.org/tests/view/
60b96b39457376eb8bcaa26e
It would be nice if a future patch could use some of the macros at
the top of the file to unify the code between the distincts SIMD
instruction sets (of course, unifying the Relu will be the challenge).
closes https://github.com/official-stockfish/Stockfish/pull/3506
No functional change
Stéphane Nicolet [Wed, 2 Jun 2021 06:10:54 +0000 (08:10 +0200)]
Makefile: better "make clean" for Windows
Make clean should be really clean on Windows.
Fixes issue https://github.com/official-stockfish/Stockfish/issues/3291
Closes https://github.com/official-stockfish/Stockfish/pull/3517
No functional change
bmc4 [Thu, 3 Jun 2021 16:52:39 +0000 (13:52 -0300)]
Introducing NodeType Root
We transform rootNode into constexpr by adding a new NodeType `Root`,
which causes a speed up.
Local test:
```
Build Tester: 1.4.7.0
Windows 10 (Version 10.0, Build 0, 64-bit Edition)
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
SafeMode: No
Running In VM: No
HyperThreading Enabled: Yes
CPU Warmup: Yes
Command Line: bench
Tests per Build: 25
ANOVA: n/a
Engine# (NPS) Speedup Sp Conf. 95% S.S.
patch (920.179,4) ---> master (906.329,2) ---> 1,528% 20.336,5 Yes No
```
---------
STC:
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 98216 W: 8348 L: 8102 D: 81766
Ptnml(0-2): 295, 6357, 35549, 6621, 286
https://tests.stockfishchess.org/tests/view/
60b797e2457376eb8bcaa0ab
Yellow LTC:
LLR: -2.95 (-2.94,2.94) <0.50,3.50>
Total: 76936 W: 2651 L: 2626 D: 71659
Ptnml(0-2): 29, 2233, 33916, 2264, 26
https://tests.stockfishchess.org/tests/view/
60b80d6d457376eb8bcaa145
closes https://github.com/official-stockfish/Stockfish/pull/3522
No functional change
xoto10 [Thu, 3 Jun 2021 18:46:55 +0000 (19:46 +0100)]
Make extra time for bestMoveInstability dependent on rootdepth.
This change allocates more base time to moves and makes the additional time added for best move instability dependent on rootdepth.
STC 10+0.1 :
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 19432 W: 1711 L: 1553 D: 16168
Ptnml(0-2): 47, 1250, 6989, 1358, 72
https://tests.stockfishchess.org/tests/view/
60b8cd41457376eb8bcaa1ad
LTC 60+0.6 :
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 22480 W: 810 L: 693 D: 20977
Ptnml(0-2): 9, 603, 9902, 714, 12
https://tests.stockfishchess.org/tests/view/
60b8e5bf457376eb8bcaa1e6
closes https://github.com/official-stockfish/Stockfish/pull/3526
Bench
4364128
Joost VandeVondele [Thu, 3 Jun 2021 11:53:28 +0000 (13:53 +0200)]
Update default net to nn-
7e66505906a6.nnue
Trained with pytorch using the master branch and recommended settings,
the data used is the previous 64B binpack enhanced with a 2B binpack
generated using an opening book of positions for with the static eval
is significantly different from d9 search.
book : https://drive.google.com/file/d/1rHcKY5rv34kwku6g89OhnE8Bkfq3UWau/view?usp=sharing
book generation: https://github.com/vondele/Stockfish/commit/
3ce43ab0c4ce09c1fc5bca5ca27a248e67fddd24
binpack : https://drive.google.com/file/d/1rHcKY5rv34kwku6g89OhnE8Bkfq3UWau/view?usp=sharing
-------
Data generation command:
generate_training_data depth 9 count
31250000 random_multi_pv 2 random_multi_pv_diff 100 random_move_max_ply 8 random_move_count 3 set_recommended_uci_options eval_limit 32000 output_file_name output.binpack book wrongNNUE.epd seed ${RANDOM}${RANDOM}
Training command:
python train.py ../../all_d9_fishd9_d8_d10_wrong_shuffle.binpack ../../all_d9_fishd9_d8_d10_wrong_shuffle.binpack --gpus 1 --threads 2 --num-workers 2 --batch-size 16384 --progress_bar_refresh_rate 300 --smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 --max_epochs=400 --seed $RANDOM --default_root_dir exp/run_5
-------
passed STC:
https://tests.stockfishchess.org/tests/view/
60b7c79a457376eb8bcaa104
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 64592 W: 6254 L: 6028 D: 52310
Ptnml(0-2): 255, 4785, 22020, 4951, 285
passed LTC:
https://tests.stockfishchess.org/tests/view/
60b85307457376eb8bcaa182
LLR: 2.96 (-2.94,2.94) <0.50,3.50>
Total: 45560 W: 1998 L: 1826 D: 41736
Ptnml(0-2): 36, 1604, 19335, 1762, 43
closes https://github.com/official-stockfish/Stockfish/pull/3521
Bench:
4364128
Stéphane Nicolet [Wed, 2 Jun 2021 06:37:00 +0000 (08:37 +0200)]
Typography change for bench
Stefan Geschwentner [Mon, 31 May 2021 18:26:05 +0000 (20:26 +0200)]
Remove formerPV variable.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 75672 W: 6546 L: 6496 D: 62630
Ptnml(0-2): 238, 5274, 26761, 5326, 237
https://tests.stockfishchess.org/tests/view/
60b349c0ec0c03148cbed055
LTC:
LLR: 2.98 (-2.94,2.94) <-2.50,0.50>
Total: 137816 W: 4676 L: 4689 D: 128451
Ptnml(0-2): 52, 4237, 60354, 4202, 63
https://tests.stockfishchess.org/tests/view/
60b38970ec0c03148cbed075
closes https://github.com/official-stockfish/Stockfish/pull/3515
Bench:
4892288
J. Oster [Mon, 31 May 2021 15:46:40 +0000 (17:46 +0200)]
Pre-initialize ss->ply
We pre-initialize ss->ply over the whole stack. There is no need
to re-assign the same value(s) over and over again while searching.
Probably a tiny speedup on longer searches.
Tested for no regression:
STC
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 25784 W: 2205 L: 2101 D: 21478
Ptnml(0-2): 62, 1660, 9368, 1716, 86
https://tests.stockfishchess.org/tests/view/
60b516c6457376eb8bca9dfa
LTC
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 26200 W: 944 L: 878 D: 24378
Ptnml(0-2): 12, 732, 11545, 800, 11
https://tests.stockfishchess.org/tests/view/
60b53652457376eb8bca9e0e
closes https://github.com/official-stockfish/Stockfish/pull/3516
No functional change.
candirufish [Mon, 31 May 2021 04:47:35 +0000 (06:47 +0200)]
Check Extension with Static Evaluation
extension for checking moves, at higher depth and more decisive positions.
stc:
LLR: 2.97 (-2.94,2.94) <-0.50,2.50>
Total: 87008 W: 7337 L: 7100 D: 72571
Ptnml(0-2): 264, 5737, 31270, 5964, 269
https://tests.stockfishchess.org/tests/view/
60b1034787a1a67ae56c47b6
ltc:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 79320 W: 2629 L: 2432 D: 74259
Ptnml(0-2): 29, 2205, 35000, 2392, 34
https://tests.stockfishchess.org/tests/view/
60b1ae0b87a1a67ae56c487c
closes https://github.com/official-stockfish/Stockfish/pull/3514
Bench:
4447112
Tomasz Sobczyk [Sat, 29 May 2021 09:40:40 +0000 (11:40 +0200)]
Fix export of the feature transformer.
PSQT export was missing.
fixes #3507
closes https://github.com/official-stockfish/Stockfish/pull/3508
No functional change
Joost VandeVondele [Sat, 29 May 2021 13:44:57 +0000 (15:44 +0200)]
Simplify NNUE / classical evaluation selection
for the new network architecture these rules can be simplified,
closer to the original PSQT difference based again.
passed STC
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 22656 W: 1979 L: 1868 D: 18809
Ptnml(0-2): 70, 1496, 8087, 1603, 72
https://tests.stockfishchess.org/tests/view/
60b24579db3c4776cb89d122
passed LTC
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 30224 W: 1015 L: 953 D: 28256
Ptnml(0-2): 4, 860, 13330, 906, 12
https://tests.stockfishchess.org/tests/view/
60b27613db3c4776cb89d145
closes https://github.com/official-stockfish/Stockfish/pull/3511
Bench:
3937626
VoyagerOne [Thu, 27 May 2021 20:03:56 +0000 (16:03 -0400)]
Remove Stat Reset at beta cutoff
STC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 63936 W: 5350 L: 5288 D: 53298
Ptnml(0-2): 184, 4295, 22954, 4345, 190
https://tests.stockfishchess.org/tests/view/
60affb4c12066fd299795c64
LTC:
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 35856 W: 1201 L: 1142 D: 33513
Ptnml(0-2): 7, 1031, 15795, 1086, 9
https://tests.stockfishchess.org/tests/view/
60b0537812066fd299795cc6
closes https://github.com/official-stockfish/Stockfish/pull/3505
bench:
3831936
Stéphane Nicolet [Wed, 26 May 2021 23:10:00 +0000 (01:10 +0200)]
Do not use lazy evaluation inside NNUE
This simplification patch implements two changes:
1. it simplifies away the so-called "lazy" path in the NNUE evaluation internals,
where we trusted the psqt head alone to avoid the costly "positional" head in
some cases;
2. it raises a little bit the NNUEThreshold1 in evaluate.cpp (from 682 to 800),
which increases the limit where we switched from NNUE eval to Classical eval.
Both effects increase the number of positional evaluations done by our new net
architecture, but the results of our tests below seem to indicate that the loss
of speed will be compensated by the gain of eval quality.
STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 26280 W: 2244 L: 2137 D: 21899
Ptnml(0-2): 72, 1755, 9405, 1810, 98
https://tests.stockfishchess.org/tests/view/
60ae73f112066fd299795a51
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 20592 W: 750 L: 677 D: 19165
Ptnml(0-2): 9, 614, 8980, 681, 12
https://tests.stockfishchess.org/tests/view/
60ae88e812066fd299795a82
closes https://github.com/official-stockfish/Stockfish/pull/3503
Bench:
3817907
Stefan Geschwentner [Wed, 26 May 2021 14:17:37 +0000 (16:17 +0200)]
Less reduction for capture/promotions.
Exclude captures/promotions at expected cut nodes (which also not a
former PV node) from LMR if a response to the first previous
opponent move.
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 288656 W: 24886 L: 24413 D: 239357
Ptnml(0-2): 900, 19738, 102578, 20213, 899
https://tests.stockfishchess.org/tests/view/
60ad505112066fd29979595b
LTC:
LLR: 2.97 (-2.94,2.94) <0.50,3.50>
Total: 31344 W: 1107 L: 975 D: 29262
Ptnml(0-2): 12, 879, 13757, 1013, 11
https://tests.stockfishchess.org/tests/view/
60adffce12066fd2997959d2
closes https://github.com/official-stockfish/Stockfish/pull/3500
Bench:
3827710
IIvec [Thu, 20 May 2021 18:02:32 +0000 (20:02 +0200)]
Simplify the thread term for reduction formula
Dependance on Threads.size() was removed Search::init() for the Reductions[] initialization.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 17376 W: 1024 L: 929 D: 15423
Ptnml(0-2): 24, 781, 6989, 864, 30
https://tests.stockfishchess.org/tests/view/
60ac110812066fd2997957dc
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 145552 W: 3656 L: 3673 D: 138223
Ptnml(0-2): 37, 3351, 66014, 3340, 34
https://tests.stockfishchess.org/tests/view/
60ac267412066fd299795825
closes https://github.com/official-stockfish/Stockfish/pull/3502
Bench
3864295
Tomasz Sobczyk [Tue, 25 May 2021 11:09:40 +0000 (13:09 +0200)]
Expose the lazy threshold for the feature transformer PSQT as a parameter.
Definition of the lazy threshold moved to evaluate.cpp where all others are.
Lazy threshold only used for real searches, not used for the "eval" call.
This preserves the purity of NNUE evaluation, which is useful to verify
consistency between the engine and the NNUE trainer.
closes https://github.com/official-stockfish/Stockfish/pull/3499
No functional change
bmc4 [Mon, 24 May 2021 11:15:04 +0000 (08:15 -0300)]
Increased reduction for captures in LMR
It now does, in LMR, an increased on reduction by 1 for captures in cut nodes.
STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 30656 W: 2565 L: 2397 D: 25694
Ptnml(0-2): 63, 2012, 11029, 2142, 82
https://tests.stockfishchess.org/tests/view/
60a96733ce8ea25a3ef04178
LTC:
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 124840 W: 4139 L: 3878 D: 116823
Ptnml(0-2): 48, 3480, 55100, 3747, 45
https://tests.stockfishchess.org/tests/view/
60a995f5ce8ea25a3ef041b7
closes https://github.com/official-stockfish/Stockfish/pull/3494
bench:
3864295
Stéphane Nicolet [Sat, 22 May 2021 17:44:15 +0000 (19:44 +0200)]
Sometimes change the (materialist, positional) balance
Our new nets output two values for the side to move in the last layer.
We can interpret the first value as a material evaluation of the
position, and the second one as the dynamic, positional value of the
location of pieces.
This patch changes the balance for the (materialist, positional) parts
of the score from (128, 128) to (121, 135) when the piece material is
equal between the two players, but keeps the standard (128, 128) balance
when one player is at least an exchange up.
Passed STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 15936 W: 1421 L: 1266 D: 13249
Ptnml(0-2): 37, 1037, 5694, 1134, 66
https://tests.stockfishchess.org/tests/view/
60a82df9ce8ea25a3ef0408f
Passed LTC:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 13904 W: 516 L: 410 D: 12978
Ptnml(0-2): 4, 374, 6088, 484, 2
https://tests.stockfishchess.org/tests/view/
60a8bbf9ce8ea25a3ef04101
closes https://github.com/official-stockfish/Stockfish/pull/3492
Bench:
3856635
bmc4 [Sat, 22 May 2021 05:47:23 +0000 (02:47 -0300)]
Tuning Search
This patch tunes constant in search.cpp
STC:
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 30648 W: 2580 L: 2410 D: 25658
Ptnml(0-2): 80, 1969, 11093, 2065, 117
https://tests.stockfishchess.org/tests/view/
60a71d3cce8ea25a3ef03fae
LTC:
LLR: 2.95 (-2.94,2.94) <0.50,3.50>
Total: 52896 W: 1776 L: 1617 D: 49503
Ptnml(0-2): 13, 1462, 23347, 1605, 21
https://tests.stockfishchess.org/tests/view/
60a794ddce8ea25a3ef0400a
closes https://github.com/official-stockfish/Stockfish/pull/3491
Bench:
4004731
bmc4 [Sat, 22 May 2021 05:41:52 +0000 (02:41 -0300)]
Simplify reduction for consecutive fails
Revert the heuristic introduced in #3184, by which we reduced more
the late sons of the root position after consecutive fail highs.
---
Before new net architecture:
STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 226336 W: 20373 L: 20500 D: 185463
Ptnml(0-2): 755, 16087, 79595, 15992, 739
https://tests.stockfishchess.org/tests/view/
609dec205085663412d08e9d
LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 67432 W: 2411 L: 2375 D: 62646
Ptnml(0-2): 33, 1944, 29714, 2004, 21
https://tests.stockfishchess.org/tests/view/
609ee30f5085663412d08fc3
---
After new net architecture:
STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 141752 W: 11591 L: 11617 D: 118544
Ptnml(0-2): 387, 9231, 51674, 9189, 395
https://tests.stockfishchess.org/tests/view/
60a4320ace8ea25a3ef03cfd
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 294072 W: 9825 L: 9950 D: 274297
Ptnml(0-2): 121, 8610, 129681, 8521, 103
https://tests.stockfishchess.org/tests/view/
60a51b5ece8ea25a3ef03dcd
---
closes https://github.com/official-stockfish/Stockfish/pull/3490
Bench:
3752892
Joost VandeVondele [Sat, 22 May 2021 05:00:53 +0000 (07:00 +0200)]
Update default net to nn-
7756374aaed3.nnue
trained with pytorch using the master branch and recommended settings,
same data set as previously used:
python train.py ../../all_d9_fishd9_d8_d10_shuffle.binpack ../../all_d9_fishd9_d8_d10_shuffle.binpack \
--gpus 1 --threads 2 --num-workers 2 --batch-size 16384 --progress_bar_refresh_rate 300 \
--smart-fen-skipping --random-fen-skipping 3 --features=HalfKAv2^ --lambda=1.0 \
--max_epochs=400 --seed $RANDOM --default_root_dir exp/run_8
passed STC:
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 21424 W: 2078 L: 1907 D: 17439
Ptnml(0-2): 80, 1512, 7385, 1627, 108
https://tests.stockfishchess.org/tests/view/
60a6c749ce8ea25a3ef03f4d
passed LTC:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 67912 W: 2851 L: 2648 D: 62413
Ptnml(0-2): 40, 2348, 28984, 2537, 47
https://tests.stockfishchess.org/tests/view/
60a722ecce8ea25a3ef03fb9
closes https://github.com/official-stockfish/Stockfish/pull/3489
Bench:
3779522
Guy Vreuls [Fri, 21 May 2021 18:22:29 +0000 (20:22 +0200)]
Compact position structures
Reorder the structures data members in position.h to reduce padding.
Passed STC:
https://tests.stockfishchess.org/tests/view/
60a8011fce8ea25a3ef04069
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 14120 W: 1214 L: 1067 D: 11839
Ptnml(0-2): 26, 857, 5161, 976, 40
---
Also tested for speed locally by Joost:
Result of 50 runs
==================
base (./stockfish.master ) =
2254919 +/- 4439
test (./stockfish.patch ) =
2274003 +/- 5278
diff = +19084 +/- 6386
==================
speedup = +0.0085
P(speedup > 0) = 1.0000
---
closes https://github.com/official-stockfish/Stockfish/pull/3488
No functional change.
Stéphane Nicolet [Tue, 18 May 2021 23:24:51 +0000 (01:24 +0200)]
Remove Tempo
The Tempo variable was introduced 10 years ago in our search because the
classical evaluation function was antisymmetrical in White and Black by design
to gain speed:
Eval(White to play) = -Eval(Black to play)
Nowadays our neural networks know which side is to play in a position when
they evaluate a position and are trained on real games, so the neural network
encodes the advantage of moving as an output of search. This patch shows that
the Tempo variable is not necessary anymore.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 33512 W: 2805 L: 2709 D: 27998
Ptnml(0-2): 80, 2209, 12095, 2279, 93
https://tests.stockfishchess.org/tests/view/
60a44ceace8ea25a3ef03d30
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 53920 W: 1807 L: 1760 D: 50353
Ptnml(0-2): 16, 1617, 23650, 1658, 19
https://tests.stockfishchess.org/tests/view/
60a477f0ce8ea25a3ef03d49
We also tried a match (20000 games) at STC using purely classical, result was neutral:
https://tests.stockfishchess.org/tests/view/
60a4eebcce8ea25a3ef03db5
Note: there are two locations left in search.cpp where we assume antisymmetry
of evaluation (in relation with a speed optimization for null moves in lines
770 and 1439), but as the values are just used for heuristic pruning this
approximation should not hurt too much because the order of magnitude is still
true most of the time.
closes https://github.com/official-stockfish/Stockfish/pull/3481
Bench:
4015864
Vizvezdenec [Wed, 19 May 2021 17:57:04 +0000 (20:57 +0300)]
Simplify usage of LMR for captures
This patch simplifies a lot of "enablers" for LMR when move is a capture or promotion.
After it we will have only 2 conditions - if node is a cutNode
or if it's an allNode that was not in PV,
so all captures or promotions wouldn't go thru LMR at any PVnodes.
passed STC
https://tests.stockfishchess.org/tests/view/
60a40117ce8ea25a3ef03ca7
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 58976 W: 4875 L: 4807 D: 49294
Ptnml(0-2): 176, 3897, 21270, 3973, 172
passed LTC
https://tests.stockfishchess.org/tests/view/
60a43ff8ce8ea25a3ef03d18
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 65272 W: 2203 L: 2165 D: 60904
Ptnml(0-2): 28, 1936, 28668, 1978, 26
closes https://github.com/official-stockfish/Stockfish/pull/3480
bench
4110764
Prokop Randáček [Wed, 12 May 2021 18:15:21 +0000 (20:15 +0200)]
Use if instead of goto
This PR inverts the if and removes goto in the generate_all function.
closes https://github.com/official-stockfish/Stockfish/pull/3461
No functional change
Fanael Linithien [Tue, 18 May 2021 17:17:59 +0000 (19:17 +0200)]
Use packed 32-bit MMX operations for updating the PSQT accumulator
This improves the speed of NNUE by a bit on old hardware that code path
is intended for, like a Pentium III 1.13 GHz:
10 repeats of "./stockfish bench 16 1 13 default depth NNUE":
Before:
54 642 504 897 cycles (± 0.12%)
62 301 937 829 instructions (± 0.03%)
After:
54 320 821 928 cycles (± 0.13%)
62 084 742 699 instructions (± 0.02%)
Speed of go depth 20 from startpos:
Before: 53103 nps
After: 53856 nps
closes https://github.com/official-stockfish/Stockfish/pull/3476
No functional change.
Yohaan Seth Nathan [Tue, 18 May 2021 18:52:59 +0000 (00:22 +0530)]
Use Markdown syntax in the readme
provide direct links to the mentioned files.
closes https://github.com/official-stockfish/Stockfish/pull/3477
No Functional Change
Vizvezdenec [Tue, 18 May 2021 07:02:20 +0000 (10:02 +0300)]
Do more continuation history based pruning
This patch increases lmrDepth threshold for continuation history based pruning in search.
This part of code for a long time was known to be really TC sensitive - decreasing
this threshold easily passed lower time controls but failed badly at LTC,
on the other hand it increase was part of a tuning that resulted
in being negative at STC but was +12 elo at 180+1.8.
After recent simplification of special conditions that sometimes
increase it from 4 to 5 it was logical to overall test at longer
time controls if 5 is better than 4 with deeper searches.
reduces strenght on STC
https://tests.stockfishchess.org/tests/view/
60a3a8bbce8ea25a3ef03c74
ELO: -2.57 +-2.0 (95%) LOS: 0.6%
Total: 20000 W: 1820 L: 1968 D: 16212
Ptnml(0-2): 68, 1582, 6836, 1458, 56
Passed LTC with STC bounds
https://tests.stockfishchess.org/tests/view/
60a027395085663412d090ce
LLR: 2.93 (-2.94,2.94) <-0.50,2.50>
Total: 175256 W: 6774 L: 6548 D: 161934
Ptnml(0-2): 91, 5808, 75604, 6034, 91
Passed VLTC with LTC bounds
https://tests.stockfishchess.org/tests/view/
60a2bccce229097940a037a7
LLR: 2.96 (-2.94,2.94) <0.50,3.50>
Total: 65736 W: 1224 L: 1092 D: 63420
Ptnml(0-2): 5, 1012, 30706, 1136, 9
closes https://github.com/official-stockfish/Stockfish/pull/3473
bench
3689330
Tomasz Sobczyk [Tue, 18 May 2021 15:36:26 +0000 (17:36 +0200)]
New NNUE architecture and net
Introduces a new NNUE network architecture and associated network parameters,
as obtained by a new pytorch trainer.
The network is already very strong at short TC, without regression at longer TC,
and has potential for further improvements.
https://tests.stockfishchess.org/tests/view/
60a159c65085663412d0921d
TC: 10s+0.1s, 1 thread
ELO: 21.74 +-3.4 (95%) LOS: 100.0%
Total: 10000 W: 1559 L: 934 D: 7507
Ptnml(0-2): 38, 701, 2972, 1176, 113
https://tests.stockfishchess.org/tests/view/
60a187005085663412d0925b
TC: 60s+0.6s, 1 thread
ELO: 5.85 +-1.7 (95%) LOS: 100.0%
Total: 20000 W: 1381 L: 1044 D: 17575
Ptnml(0-2): 27, 885, 7864, 1172, 52
https://tests.stockfishchess.org/tests/view/
60a2beede229097940a03806
TC: 20s+0.2s, 8 threads
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 34272 W: 1610 L: 1452 D: 31210
Ptnml(0-2): 30, 1285, 14350, 1439, 32
https://tests.stockfishchess.org/tests/view/
60a2d687e229097940a03c72
TC: 60s+0.6s, 8 threads
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 45544 W: 1262 L: 1214 D: 43068
Ptnml(0-2): 12, 1129, 20442, 1177, 12
The network has been trained (by vondele) using the https://github.com/glinscott/nnue-pytorch/ trainer (started by glinscott),
specifically the branch https://github.com/Sopel97/nnue-pytorch/tree/experiment_56.
The data used are in 64 billion positions (193GB total) generated and scored with the current master net
d8: https://drive.google.com/file/d/1hOOYSDKgOOp38ZmD0N4DV82TOLHzjUiF/view?usp=sharing
d9: https://drive.google.com/file/d/1VlhnHL8f-20AXhGkILujnNXHwy9T-MQw/view?usp=sharing
d10: https://drive.google.com/file/d/1ZC5upzBYMmMj1gMYCkt6rCxQG0GnO3Kk/view?usp=sharing
fishtest_d9: https://drive.google.com/file/d/1GQHt0oNgKaHazwJFTRbXhlCN3FbUedFq/view?usp=sharing
This network also contains a few architectural changes with respect to the current master:
Size changed from 256x2-32-32-1 to 512x2-16-32-1
~15-20% slower
~2x larger
adds a special path for 16 valued ClippedReLU
fixes affine transform code for 16 inputs/outputs, buy using InputDimensions instead of PaddedInputDimensions
this is safe now because the inputs are processed in groups of 4 in the current affine transform code
The feature set changed from HalfKP to HalfKAv2
Includes information about the kings like HalfKA
Packs king features better, resulting in 8% size reduction compared to HalfKA
The board is flipped for the black's perspective, instead of rotated like in the current master
PSQT values for each feature
the feature transformer now outputs a part that is fowarded directly to the output and allows learning piece values more directly than the previous network architecture. The effect is visible for high imbalance positions, where the current master network outputs evaluations skewed towards zero.
8 PSQT values per feature, chosen based on (popcount(pos.pieces()) - 1) / 4
initialized to classical material values on the start of the training
8 subnetworks (512x2->16->32->1), chosen based on (popcount(pos.pieces()) - 1) / 4
only one subnetwork is evaluated for any position, no or marginal speed loss
A diagram of the network is available: https://user-images.githubusercontent.com/
8037982/
118656988-
553a1700-b7eb-11eb-82ef-
56a11cbebbf2.png
A more complete description: https://github.com/glinscott/nnue-pytorch/blob/master/docs/nnue.md
closes https://github.com/official-stockfish/Stockfish/pull/3474
Bench:
3806488
Stéphane Nicolet [Mon, 17 May 2021 07:13:34 +0000 (09:13 +0200)]
Small clean-ups
- Comment for Countemove pruning -> Continuation history
- Fix comment in input_slice.h
- Shorter lines in Makefile
- Comment for scale factor
- Fix comment for pinners in see_ge()
- Change Thread.id() signature to size_t
- Trailing space in reprosearch.sh
- Add Douglas Matos Gomes to the AUTHORS file
- Introduce comment for undo_null_move()
- Use Stockfish coding style for export_net()
- Change date in AUTHORS file
closes https://github.com/official-stockfish/Stockfish/pull/3416
No functional change
Vizvezdenec [Fri, 14 May 2021 21:55:45 +0000 (00:55 +0300)]
Simplification for countermoves based pruning
Simplify away two extra conditions in countermoves based pruning.
These conditions (both of them) were introduced quite a long time ago
via speculative LTCs and seem to no longer bring any benefit.
passed STC
https://tests.stockfishchess.org/tests/view/
609e81f35085663412d08f31
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 28488 W: 2487 L: 2382 D: 23619
Ptnml(0-2): 87, 1919, 10123, 2032, 83
passed LTC
https://tests.stockfishchess.org/tests/view/
609e9c085085663412d08f59
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 33176 W: 1219 L: 1155 D: 30802
Ptnml(0-2): 13, 1036, 14423, 1106, 10
closes https://github.com/official-stockfish/Stockfish/pull/3468
Bench:
4749514
bmc4 [Fri, 14 May 2021 02:47:41 +0000 (23:47 -0300)]
Simplify LMR rules for statScore
We simplify two parts of LMR which seem not to bring strength anymore.
---
Individual Tests:
https://tests.stockfishchess.org/tests/view/
609d1cc15085663412d0856a
https://tests.stockfishchess.org/tests/view/
609cb0cc7746e3dc74ffae8d
https://tests.stockfishchess.org/tests/view/
609d1c9f5085663412d08568
---
LTC:
LLR: 2.97 (-2.94,2.94) <-2.50,0.50>
Total: 84184 W: 3093 L: 3066 D: 78025
Ptnml(0-2): 47, 2755, 36458, 2788, 44
https://tests.stockfishchess.org/tests/view/
609d84615085663412d08e2f
---
While at it, we also update the Elo estimate of the previous rule in LMR, see:
https://tests.stockfishchess.org/tests/view/
609a933c3a33eb67a844f7ca
https://tests.stockfishchess.org/tests/view/
609a959c3a33eb67a844f7d5
https://tests.stockfishchess.org/tests/view/
609afff73a33eb67a844f870
---
closes https://github.com/official-stockfish/Stockfish/pull/3464
Bench:
4156523
bmc4 [Thu, 13 May 2021 14:12:56 +0000 (11:12 -0300)]
Remove early return in Probcut code
We simplify away early return in ProbCut, as it seems not to bring any strength anymore.
STC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 42632 W: 3705 L: 3617 D: 35310
Ptnml(0-2): 123, 2947, 15110, 2991, 145
https://tests.stockfishchess.org/tests/view/
609c49da7746e3dc74ffae02
LTC:
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 35384 W: 1314 L: 1251 D: 32819
Ptnml(0-2): 11, 1130, 15355, 1177, 19
https://tests.stockfishchess.org/tests/view/
609c71467746e3dc74ffae47
---
While at it, we also update the Elo estimate of ProbCut
(see https://tests.stockfishchess.org/tests/view/
609bfb597746e3dc74ffabe3).
closes https://github.com/official-stockfish/Stockfish/pull/3462
bench:
3764662
Unai Corzo [Fri, 14 May 2021 15:35:32 +0000 (17:35 +0200)]
Remove BoolConditions from tuning code
Remove BoolConditions from tuning code, as the feature does not work
and the code has not be touched in years.
No functional change
bmc4 [Wed, 12 May 2021 12:38:26 +0000 (09:38 -0300)]
Simplify LMR rule for non-checking captures
We simplify away the complicated rule in LMR for "non-checking captures
likely to be bad", as it seems not to bring any strength anymore.
STC:
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 55256 W: 4972 L: 4897 D: 45387
Ptnml(0-2): 177, 3976, 19234, 4077, 164
https://tests.stockfishchess.org/tests/view/
609adf3b3a33eb67a844f842
LTC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 10344 W: 437 L: 353 D: 9554
Ptnml(0-2): 1, 322, 4449, 392, 8
https://tests.stockfishchess.org/tests/view/
609b3dfa3a33eb67a844f88e
--
While at it, we also update the Elo estimate of the previous rule in LMR
(see https://tests.stockfishchess.org/tests/view/
609af2a63a33eb67a844f867).
closes https://github.com/official-stockfish/Stockfish/pull/3460
Bench:
3840688
EntityFX [Mon, 12 Apr 2021 10:42:35 +0000 (13:42 +0300)]
E2K: added support for MCST Elbrus 2000 CPU architecture
e2k (Elbrus 2000) - this is a VLIW/EPIC architecture,
the like Intel Itanium (IA-64) architecture.
The architecture has half native / half software support
for most Intel/AMD SIMD (e.g. MMX/SSE/SSE2/SSE3/SSSE3/SSE4.1/SSE4.2/AES/AVX/AVX2 & 3DNow!/SSE4a/XOP/FMA4) via intrinsics.
https://en.wikipedia.org/wiki/Elbrus_2000
closes https://github.com/official-stockfish/Stockfish/pull/3425
No functional change
bmc4 [Sun, 9 May 2021 19:33:41 +0000 (16:33 -0300)]
Remove coordination between searching threads
In summary, this revert #2204, as it seems not to bring any strength anymore, so it's no long needed.
STC (5+0.05 @ 8 threads):
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 105728 W: 6406 L: 6393 D: 92929
Ptnml(0-2): 154, 5479, 41599, 5464, 168
https://tests.stockfishchess.org/tests/view/
6096994095e7f1852abd3154
LTC (20+0.2 @ 8 threads):
LLR: 2.96 (-2.94,2.94) <-2.50,0.50>
Total: 26336 W: 774 L: 712 D: 24850
Ptnml(0-2): 9, 641, 11810, 695, 13
https://tests.stockfishchess.org/tests/view/
6097c62995e7f1852abd31e8
closes https://github.com/official-stockfish/Stockfish/pull/3459
No functional change.
bmc4 [Tue, 11 May 2021 14:04:02 +0000 (11:04 -0300)]
Simplify LMR
as it seems not to bring any strength and thus is no longer needed.
Tests for updating elo estimates:
https://tests.stockfishchess.org/tests/view/
6099ff123a33eb67a844f789
https://tests.stockfishchess.org/tests/view/
60953e6695e7f1852abd305b
Individual simplification tests:
https://tests.stockfishchess.org/tests/view/
6098cfc73a33eb67a844f6a1
https://tests.stockfishchess.org/tests/view/
6095539495e7f1852abd308b
LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 96984 W: 3624 L: 3608 D: 89752
Ptnml(0-2): 45, 3222, 41939, 3244, 42
https://tests.stockfishchess.org/tests/view/
6099921a3a33eb67a844f74f
closes https://github.com/official-stockfish/Stockfish/pull/3458
bench:
3836428
Tomasz Sobczyk [Fri, 7 May 2021 10:24:12 +0000 (12:24 +0200)]
Exporting the currently loaded network file
This PR adds an ability to export any currently loaded network.
The export_net command now takes an optional filename parameter.
If the loaded net is not the embedded net the filename parameter is required.
Two changes were required to support this:
* the "architecture" string, which is really just a some kind of description in the net, is now saved into netDescription on load and correctly saved on export.
* the AffineTransform scrambles weights for some architectures and sparsifies them, such that retrieving the index is hard. This is solved by having a temporary scrambled<->unscrambled index lookup table when loading the network, and the actual index is saved for each individual weight that makes it to canSaturate16. This increases the size of the canSaturate16 entries by 6 bytes.
closes https://github.com/official-stockfish/Stockfish/pull/3456
No functional change
Vizvezdenec [Wed, 5 May 2021 16:03:20 +0000 (19:03 +0300)]
Cleanup of likelyFailLow logic
This patch broadens and simplifies definition of PvNode that is likely to fail low.
New definition can be described as following "If node was already researched
at depth >= current depth and failed low there" which is more logical than the
previous version and takes less space + allows to not recompute it every time during move loop.
Passed simplification STC
https://tests.stockfishchess.org/tests/view/
609148bf95e7f1852abd2e82
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 20128 W: 1865 L: 1751 D: 16512
Ptnml(0-2): 63, 1334, 7165, 1430, 72
Passed simplification LTC
https://tests.stockfishchess.org/tests/view/
6091691295e7f1852abd2e8b
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 95128 W: 3498 L: 3481 D: 88149
Ptnml(0-2): 41, 2956, 41549, 2981, 37
closes https://github.com/official-stockfish/Stockfish/pull/3455
Bench:
3933037
Tomasz Sobczyk [Sun, 2 May 2021 16:50:09 +0000 (18:50 +0200)]
Add an UCI level command "export_net".
This command writes the embedded net to the file `EvalFileDefaultName`.
If there is no embedded net the command does nothing.
fixes #3453
closes https://github.com/official-stockfish/Stockfish/pull/3454
No functional change
Unai Corzo [Sat, 1 May 2021 08:18:57 +0000 (10:18 +0200)]
Simplify check extension
Simplify check extension, as it seems not to bring any strength and thus is no longer needed.
STC https://tests.stockfishchess.org/tests/view/
608c18e995e7f1852abd2b81
LLR: 2.94 (-2.94,2.94) <-2.50,0.50>
Total: 54544 W: 4891 L: 4815 D: 44838
Ptnml(0-2): 186, 3889, 19081, 3895, 221
LTC https://tests.stockfishchess.org/tests/view/
608c6ab195e7f1852abd2bc6
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 51008 W: 1845 L: 1794 D: 47369
Ptnml(0-2): 31, 1591, 22206, 1648, 28
closes https://github.com/official-stockfish/Stockfish/pull/3452
bench:
3993071
Joost VandeVondele [Thu, 29 Apr 2021 06:18:37 +0000 (08:18 +0200)]
Add some more information on the UCI protocol
Improve README.md: provide a link to the protocol,
and document some non-standard options.
fixes https://github.com/official-stockfish/Stockfish/issues/3446
closes https://github.com/official-stockfish/Stockfish/pull/3450
No functional change
xoto10 [Sat, 24 Apr 2021 13:46:01 +0000 (14:46 +0100)]
Change tempo with time and threads
Introduce variable tempo for nnue depending on logarithm of estimated
strength, where strength is the product of time and number of threads.
The original idea here was that NNUE is best with a slightly different
tempo value to classical, since its style of play is slightly different.
It turns out that the best tempo for NNUE varies with strength of play,
so a formula is used which gives about 19 for STC and 24 for LTC under
current fishtest settings.
STC 10+0.1:
LLR: 2.94 (-2.94,2.94) {-0.20,1.10}
Total: 120816 W: 11155 L: 10861 D: 98800
Ptnml(0-2): 406, 8728, 41933, 8848, 493
https://tests.stockfishchess.org/tests/view/
60735b3a8141753378960534
LTC 60+0.6:
LLR: 2.94 (-2.94,2.94) {0.20,0.90}
Total: 35688 W: 1392 L: 1234 D: 33062
Ptnml(0-2): 23, 1079, 15473, 1255, 14
https://tests.stockfishchess.org/tests/view/
6073ffbc814175337896057f
Passed non-regression SMP test at LTC 20+0.2 (8 threads):
LLR: 2.95 (-2.94,2.94) {-0.70,0.20}
Total: 11008 W: 317 L: 267 D: 10424
Ptnml(0-2): 2, 245, 4962, 291, 4
https://tests.stockfishchess.org/tests/view/
60749ea881417533789605a4
closes https://github.com/official-stockfish/Stockfish/pull/3426
Bench
4075325
bmc4 [Tue, 27 Apr 2021 22:59:46 +0000 (19:59 -0300)]
Simplify pawn moves generator
This patch simplifies QUIET_CHECKS pawn move generator by merging discovery check
move generator with direct check move generator. It also simplifies emptySquares
instantiation. In addition, I added a comment in generate_moves() to clarify Check
branches.
STC:
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 112648 W: 9952 L: 9945 D: 92751
Ptnml(0-2): 369, 7682, 40195, 7729, 349
https://tests.stockfishchess.org/tests/view/
6088226895e7f1852abd2978
LTC:
LLR: 2.93 (-2.94,2.94) <-2.50,0.50>
Total: 74656 W: 2797 L: 2765 D: 69094
Ptnml(0-2): 38, 2328, 32554, 2380, 28
https://tests.stockfishchess.org/tests/view/
60884e5095e7f1852abd2994
closes https://github.com/official-stockfish/Stockfish/pull/3447
No functional change
lonfom169 [Sun, 25 Apr 2021 00:37:47 +0000 (21:37 -0300)]
More extensions if SE search is very low.
More extensions for non-PV nodes if value from singular extension search is significantly below singularBeta.
Passed STC:
LLR: 2.97 (-2.94,2.94) <-0.50,2.50>
Total: 25064 W: 2334 L: 2162 D: 20568
Ptnml(0-2): 82, 1720, 8768, 1868, 94
https://tests.stockfishchess.org/tests/view/
6084ba7995e7f1852abd27e3
Passed LTC:
LLR: 2.94 (-2.94,2.94) <0.50,3.50>
Total: 67136 W: 2644 L: 2450 D: 62042
Ptnml(0-2): 46, 2134, 28990, 2376, 22
https://tests.stockfishchess.org/tests/view/
6084d79195e7f1852abd27ee
closes https://github.com/official-stockfish/Stockfish/pull/3445
Bench:
4075325
Stefan Geschwentner [Sun, 25 Apr 2021 05:09:35 +0000 (07:09 +0200)]
Thread based reduction tweak.
For PV nodes at the first two plies no reductions are done for each fourth thread.
STC (8 threads):
LLR: 2.94 (-2.94,2.94) <-0.50,2.50>
Total: 53992 W: 3334 L: 3167 D: 47491
Ptnml(0-2): 64, 2713, 21285, 2860, 74
https://tests.stockfishchess.org/tests/view/
6083b2d695e7f1852abd277a
LTC (8 threads):
LLR: 2.93 (-2.94,2.94) <0.50,3.50>
Total: 64888 W: 1888 L: 1725 D: 61275
Ptnml(0-2): 14, 1556, 29146, 1709, 19
https://tests.stockfishchess.org/tests/view/
6084249595e7f1852abd2795
closes https://github.com/official-stockfish/Stockfish/pull/3443
No functional change (for one thread)
Tomasz Sobczyk [Sat, 24 Apr 2021 13:08:11 +0000 (15:08 +0200)]
Cleanup and simplify NNUE code.
A lot of optimizations happend since the NNUE was introduced
and since then some parts of the code were left unused. This
got to the point where asserts were have to be made just to
let people know that modifying something will not have any
effects or may even break everything due to the assumptions
being made. Removing these parts removes those inexisting
"false dependencies". Additionally:
* append_changed_indices now takes the king pos and stateinfo
explicitly, no more misleading pos parameter
* IndexList is removed in favor of a generic ValueList.
Feature transformer just instantiates the type it needs.
* The update cost and refresh requirement is deferred to the
feature set once again, but now doesn't go through the whole
FeatureSet machinery and just calls HalfKP directly.
* accumulator no longer has a singular dimension.
* The PS constants and the PieceSquareIndex array are made local
to the HalfKP feature set because they are specific to it and
DO differ for other feature sets.
* A few names are changed to more descriptive
Passed STC non-regression:
https://tests.stockfishchess.org/tests/view/
608421dd95e7f1852abd2790
LLR: 2.95 (-2.94,2.94) <-2.50,0.50>
Total: 180008 W: 16186 L: 16258 D: 147564
Ptnml(0-2): 587, 12593, 63725, 12503, 596
closes https://github.com/official-stockfish/Stockfish/pull/3441
No functional change
bmc4 [Thu, 22 Apr 2021 17:07:37 +0000 (14:07 -0300)]
Merge all move generators
Merging `generate<EVASIONS>` and `generate<QUIET_CHECKS>` into `generate_all()`.
verified to yield correct perft results, even though bench changes due to different order of generated moves.
No regresion playing games:
passed STC:
LLR: 2.94 (-2.94,2.94) {-1.00,0.20}
Total: 161800 W: 14585 L: 14624 D: 132591
Ptnml(0-2): 577, 11681, 56451, 11586, 605
https://tests.stockfishchess.org/tests/view/
606532732b2df919fd5f026d
passed LTC:
LLR: 2.98 (-2.94,2.94) {-0.70,0.20}
Total: 188504 W: 6906 L: 6961 D: 174637
Ptnml(0-2): 87, 6272, 81610, 6175, 108
https://tests.stockfishchess.org/tests/view/
6065b0772b2df919fd5f02ae
closes https://github.com/official-stockfish/Stockfish/pull/3418
Bench:
4536129
Tomasz Sobczyk [Mon, 19 Apr 2021 17:50:19 +0000 (19:50 +0200)]
Unify naming convention of the NNUE code
matches the rest of the stockfish code base
closes https://github.com/official-stockfish/Stockfish/pull/3437
No functional change
dsmsgms [Mon, 12 Apr 2021 15:06:22 +0000 (12:06 -0300)]
Use classical eval for Bishop vs Pawns
NNUE evaluation is incapable of recognizing trivially drawn bishop endgames
(the wrong-colored rook pawn), which are in fact ubiquitous and stock standard
in chess analysis. Switching off NNUE evaluation in KBPs vs KPs endgames is
a measure that stops Stockfish from trading down to a drawn version of these
endings when we presumably have advantage. The patch is able to edge over master
in endgame positions.
Patch tested for Elo gain with the "endgame.epd" book, and verified for
non-regression with our usual book (see the pull request for details).
STC:
LLR: 2.93 (-2.94,2.94) {-0.20,1.10}
Total: 33232 W: 6655 L: 6497 D: 20080
Ptnml(0-2): 4, 2342, 11769, 2494, 7
https://tests.stockfishchess.org/tests/view/
6074a52981417533789605b8
LTC:
LLR: 2.93 (-2.94,2.94) {0.20,0.90}
Total: 159056 W: 29799 L: 29378 D: 99879
Ptnml(0-2): 7, 9004, 61085, 9425, 7
https://tests.stockfishchess.org/tests/view/
6074c39a81417533789605ca
Closes https://github.com/official-stockfish/Stockfish/pull/3427
Bench:
4503918
blah
Tomasz Sobczyk [Tue, 13 Apr 2021 18:02:28 +0000 (20:02 +0200)]
Documentation patch: AppendChangedIndices
Clarify the assumptions on the position passed to the AppendChangedIndices().
Closes https://github.com/official-stockfish/Stockfish/pull/3428
No functional change
Vizvezdenec [Wed, 14 Apr 2021 18:25:18 +0000 (21:25 +0300)]
Simplification: last capture extension
The code for last capture extension can be removed in current master.
Passed STC
LLR: 2.95 (-2.94,2.94) {-1.00,0.20}
Total: 85024 W: 7754 L: 7707 D: 69563
Ptnml(0-2): 293, 5991, 29914, 6004, 310
https://tests.stockfishchess.org/tests/view/
607690f1814175337896068f
Passed LTC
LLR: 2.96 (-2.94,2.94) {-0.70,0.20}
Total: 39880 W: 1503 L: 1453 D: 36924
Ptnml(0-2): 17, 1281, 17293, 1333, 16
https://tests.stockfishchess.org/tests/view/
6076ccbe814175337896069e
Closes https://github.com/official-stockfish/Stockfish/pull/3430
Bench:
4202264
Stéphane Nicolet [Thu, 15 Apr 2021 09:18:38 +0000 (11:18 +0200)]
Revert previous patch
Revert the previous patch about move generation, as it unexpectedly
changed the bench. Better to take the time to understand the issue.
Bench:
4191632