Michael Chaly [Mon, 18 Dec 2023 17:34:35 +0000 (20:34 +0300)]
Adjust return value in qsearch after fail high
Instead of returning strict fail soft fail high return value between value from
search and beta (somewhat by analogy to futility pruning and probcut).
This seems to be somewhat depth sensitive heuristic which performed much worse
at LTC while performing much better at STC if it is more aggressive, passed
version is the least aggressive one.
Michael Chaly [Thu, 14 Dec 2023 22:51:52 +0000 (01:51 +0300)]
Adjust stand pat in qsearch on pv nodes
Instead of immediately returning a fail high do this only at non-pv nodes, for
pv nodes adjust bestValue to value between alpha and beta and continue
searching. Idea is to do it the same way as it's done in search where we don't
return positive beta cutoffs after ttHits / zero window search at PvNodes and
instead fully search lines.
Two issues had to be resolved:
- in rare cases it set alpha to the same value as beta and thus broke some asserts;
- messed up with returning tb win values.
Muzhen Gaming [Wed, 13 Dec 2023 11:05:37 +0000 (19:05 +0800)]
Remove UCI_AnalyseMode Option
Simplify away the useless option, as documented: "An option handled by your GUI.
This currently doesn't do anything."
The option was originally added with the introduction of contempt
(https://github.com/official-stockfish/Stockfish/commit/e9aeaad05266ca557a9496b5a17b4c5f82f0e946),
but it is now no longer used.
FauziAkram [Tue, 12 Dec 2023 13:38:24 +0000 (16:38 +0300)]
Remove SlowMover Option
The SlowMover option allows users to modify the timeLeft variant, impacting the
engine's time management. However, this feature, while theoretically flexible,
doesn't offer substantial benefits. Instead, it introduces the risk of
non-experienced users altering values without a clear understanding of the
effects, potentially leading to a weaker engine.
The vast majority of SF users don't use it anyway, and based on tests conducted
by fauzi several months ago suggest that changing it would only lose Elo.
WangXiang [Sun, 10 Dec 2023 15:04:32 +0000 (23:04 +0800)]
Add loongarch64 support
Adding support for LoongArch64 architecture. Tested on Loongson 3A6000 EVB
Board. Since Loongson's SIMD extended instruction set
([LSX](https://gcc.gnu.org/onlinedocs/gcc/LoongArch-SX-Vector-Intrinsics.html),
[LASX](https://gcc.gnu.org/onlinedocs/gcc/LoongArch-ASX-Vector-Intrinsics.html))
is already supported by GCC, more optimizations are being developed.
Here's the benchmark result for Loongson 3A6000 (4c8t, 2.5Ghz) without SIMD
optimizations.
```
Total time (ms) : 17903
Nodes searched : 1244386
Nodes/second : 69507
```
peregrineshahin [Sun, 3 Dec 2023 21:39:41 +0000 (00:39 +0300)]
Track seldepth in qsearch too
Sometimes if we count the reported PV length, it turns out to be longer than the
selective depth reported. This fixes this behavior by applying the selective
depth to qsearch since we do report PVs from it as well.
Shahin M. Shahin [Fri, 17 Nov 2023 15:25:50 +0000 (18:25 +0300)]
Fix TB score output in UCI without using TB
This is a rewrite of the fix introduced for
https://github.com/official-stockfish/Stockfish/issues/4413 in
https://github.com/official-stockfish/Stockfish/pull/4591 by @windfishballad it
targets only the relevant part of this issue that returns TB scores (CP 20000)
without using TB due to the downgrading of potentially false mates from the TT
to an optimal TB score.
the difference is that it is a much clearer code that introduces a separate
TB_VALUE constant to account for a correct distance from the TB_VALUE with
MAX_PLY.
the originally posted position in the issue does not trigger the problem
anymore, so here is a new position to test:
```
position fen 3k4/8/8/8/8/8/3BN3/3K4 w - - 0 1
go infinite
```
Muzhen Gaming [Sat, 25 Nov 2023 14:16:56 +0000 (22:16 +0800)]
Simplify move history reduction
Recent VLTC search tuning has suggested that the depth limit can be increased
by a lot. This patch simplifies away the depth-based bonus from statScore
reduction, making the divisor a constant.
We use following line to clamp the search depth in some range:
Depth d = std::clamp(newDepth - r, 1, newDepth + 1);
Through negative extension its possible that the maximum value becomes smaller than the minimum value but then the behavior is undefined (see https://en.cppreference.com/w/cpp/algorithm/clamp). So replace this line with a safe implementation.
Remark:
We have in recent master already one line where up to 3 negative extensions are possible which could trigger this undefined behavior but this can only be happen for completed depth > 24 so its not discovered by our default bench. Recent negative extension tests by @fauzi shows then this undefined behavior with wrong bench numbers.
- updates the SDE action to v2.2
- removes the linux x86-32 builds, which were almost unused,
and the build process under SDE started failing recently,
possibly related to glibc update (The futex facility returned an unexpected error code.)
Double weight of pawn history for quiet move ordering.
I measured on my 1000 position bench the average additional added pawn history per depth.
This shows on average negative value with even smaller values with increaing depth.
A linear regression against depth get following formula:
-1960 - 130 * depth
For compensation add this to the used sort limit to maintain roughly the same proportion of sorted quiet moves.
Remarks:
1. using no compensation failed here https://tests.stockfishchess.org/tests/view/6547664f136acbc5735265f0
2. using only the compensation failed at LTC:
passed STC: https://tests.stockfishchess.org/tests/view/65477457136acbc5735266f8
failed LTC: https://tests.stockfishchess.org/tests/view/65487fc8136acbc573527d1c
FauziAkram [Sat, 4 Nov 2023 14:19:08 +0000 (17:19 +0300)]
Tweaking the futility pruning formula
Huge credit goes also to candirufish,
as the idea was first tried by him, and then tuned by me at multiple phases.
Tweaking the futility pruning formula to be a bit more selective about when pruning is applied.
Adjust the value added to the static eval based on the bestValue relative to ss->staticEval. If bestValue is significantly lower, we add a larger value.
The recently commit 'Rewarding Quiet Moves that Enable Razoring' add a history update if razoring. But its contains also many tuned values all over the search. Following tests shows that the tuned values and not the added history update is responsible for the elo gain. So remove later.
FauziAkram [Fri, 27 Oct 2023 10:28:55 +0000 (13:28 +0300)]
Rewarding Quiet Moves that Enable Razoring
The main idea of the patch comes from @peregrineshahin :
https://tests.stockfishchess.org/tests/view/65205363ac57711436728781
Another small idea (tuning) comes from @Vizvezdenec
https://tests.stockfishchess.org/tests/view/652e071ade6d262d08d318f4 And a long
phases of tuning and tests was done by me in order to make the patch be able to
pass both tests.
The idea, as mentioned by Peregrine is that in our standard code, if no best
move found after searching all moves, we give a bonus to the previous move that
caused the fail high. So in razoring we assume no bestmove will be found so we
might as well do the same.
Michael Chaly [Fri, 27 Oct 2023 15:19:31 +0000 (17:19 +0200)]
Introduce pawn structure based history
Original idea by Seer chess engine https://github.com/connormcmonigle/seer-nnue,
coding done by @Disservin, code refactoring done by @locutus2 to match the style
of other histories.
This patch introduces pawn structure based history, which assings moves values
based on last digits of pawn structure hash and piece type of moved piece and
landing square of the move. Idea is that good places for pieces are quite often
determined by pawn structure of position. Used in 3 different places
- sorting of quiet moves, sorting of quiet check evasions and in history based
pruning in search.
Muzhen Gaming [Sat, 21 Oct 2023 09:01:45 +0000 (17:01 +0800)]
Time management improvements
1. Tune time management parameters.
2. Scale the optimum time and maximum time parameters based on the amount of
time left, using a logarithmic scale.
Many acknowledgements to @FauziAkram for tuning the parameters and for the
original idea (see
https://tests.stockfishchess.org/tests/view/652f0356de6d262d08d333c5).
Disservin [Sun, 22 Oct 2023 18:20:53 +0000 (20:20 +0200)]
remove blank line between function and it's description
- remove the blank line between the declaration of the function and it's
comment, leads to better IDE support when hovering over a function to see it's
description
- remove the unnecessary duplication of the function name in the functions
description
- slightly refactored code for lsb, msb in bitboard.h There are still a few
things we can be improved later on, move the description of a function where
it was declared (instead of implemented) and add descriptions to functions
which are behind macros ifdefs
Disservin [Sun, 22 Oct 2023 14:43:33 +0000 (16:43 +0200)]
use expanded variables for shell commands
Performance improvement for the shell commands in the Makefile.
By using expanded variables, the shell commands are only
evaluated once, instead of every time they are used.
Taras Vuk [Sun, 22 Oct 2023 06:36:43 +0000 (08:36 +0200)]
Simplify futilityBase formula
This patch replaces std::min(ss->staticEval, bestValue) with ss->staticEval in the futilityBase formula.
Original idea by Vizvezdenec: https://tests.stockfishchess.org/tests/view/64ce66795b17f7c21c0d85f3
Disservin [Sat, 21 Oct 2023 09:40:56 +0000 (11:40 +0200)]
add clang-format
This introduces clang-format to enforce a consistent code style for Stockfish.
Having a documented and consistent style across the code will make contributing easier
for new developers, and will make larger changes to the codebase easier to make.
To facilitate formatting, this PR includes a Makefile target (`make format`) to format the code,
this requires clang-format (version 17 currently) to be installed locally.
Installing clang-format is straightforward on most OS and distros
(e.g. with https://apt.llvm.org/, brew install clang-format, etc), as this is part of quite commonly
used suite of tools and compilers (llvm / clang).
Additionally, a CI action is present that will verify if the code requires formatting,
and comment on the PR as needed. Initially, correct formatting is not required, it will be
done by maintainers as part of the merge or in later commits, but obviously this is encouraged.
This patch is a simplification and a fix to dealing with null moves scores that returns proven mates or TB scores by preventing 'null move pruning' if the nullvalue is in that range.
Current solution downgrades nullValues on the non-PV node but the value can be used in a transposed PV-node to the same position afterwards (Triangulation), the later is prone to propagate a wrong score (96.05) to root that will not be refuted unless we search further.
Score of (96.05) can be obtained be two methods,
maxim static-eval returned on Pv update (mostly qSearch)
this downgrade (clamp) in NMP
and theoretically can happen with or without TBs but the second scenario is more dangerous than the first.
This fixes the reproducible case in very common scenarios with TBs as shown in the debugging at discord.
Michael Chaly [Sat, 14 Oct 2023 14:41:41 +0000 (17:41 +0300)]
Use more continuation histories.
This patch allows stats updates and movepicker bonuses for continuation history 3 plies deep - so counter counter move.
Updates and movepicker usage are done with 1/4 multiplier compared to other histories.
Simplify collection of bad moves for history updates.
1. collect only the first 32 moves searched and ignore the rest. So late bad moves get no further negative history updates.
2. collect now for quiet moves also at most 32 bad moves
mstembera [Sun, 1 Oct 2023 06:12:02 +0000 (23:12 -0700)]
Optimize the most common update accumalator cases w/o tiling
In the most common case where we only update a single state
it's faster to not use temporary accumulation registers and tiling.
(Also includes a couple of small cleanups.)
A simpler version
https://tests.stockfishchess.org/tests/view/65190dfacff46e538ee00155
also passed but this version is stronger still
https://tests.stockfishchess.org/tests/view/6519b95fcff46e538ee00fa2
This is a later epoch from the same experiment that led to the previous
master net. In training stage 6, max-epoch was raised to 1,200 near the
end of the first 1,000 epochs.
For more details, see https://github.com/official-stockfish/Stockfish/pull/4795
Local elo at 25k nodes per move (vs. L1-2048 nn-1ee1aba5ed4c.nnue)
ep1079 : 15.6 +/- 1.2
Linmiao Xu [Fri, 12 May 2023 22:07:20 +0000 (18:07 -0400)]
Update NNUE architecture to SFNNv8: L1-2560 nn-ac1dbea57aa3.nnue
Creating this net involved:
- a 6-stage training process from scratch. The datasets used in stages 1-5 were fully minimized.
- permuting L1 weights with https://github.com/official-stockfish/nnue-pytorch/pull/254
A strong epoch after each training stage was chosen for the next. The 6 stages were:
```
1. 400 epochs, lambda 1.0, default LR and gamma
UHOx2-wIsRight-multinet-dfrc-n5000 (135G)
nodes5000pv2_UHO.binpack
data_pv-2_diff-100_nodes-5000.binpack
wrongIsRight_nodes5000pv2.binpack
multinet_pv-2_diff-100_nodes-5000.binpack
dfrc_n5000.binpack
5. 960 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28
Increased max-epoch to 960 near the end of the first 800 epochs 5af11540bbfe dataset: https://github.com/official-stockfish/Stockfish/pull/4635
6. 1000 epochs, end-lambda 0.7, LR 4.375e-4, gamma 0.995, skip 28
Increased max-epoch to 1000 near the end of the first 800 epochs 1ee1aba5ed dataset: https://github.com/official-stockfish/Stockfish/pull/4782
```
After removing classic evaluation VALUE_KNOWN_WIN is not anymore returned explicit evaluation. So remove and replace it with VALUE_TB_WIN_IN_MAX_PLY.
Measurement on my big bench (bench 16 1 16 pos1000.fen) verifies that at least with current net the calculated evaluation lies always in the open interval (-VALUE_KNOWN_WIN, VALUE_KNOWN_WIN).
So i consider this a non-functional change. But to be safe i tested this also at LTC as requested by Stephane Nicolet.
The commit adds a CI workflow that uses the included-what-you-use (IWYU)
tool to check for missing or superfluous includes in .cpp files and
their corresponding .h files. This means that some .h files (especially
in the nnue folder) are not checked yet.
The CI setup looks like this:
- We build IWYU from source to include some yet unreleased fixes.
This IWYU version targets LLVM 17. Thus, we get the latest release
candidate of LLVM 17 from LLVM's nightly packages.
- The Makefile now has an analyze target that just build the object
files (without linking)
- The CI uses the analyze target with the IWYU tool as compiler to
analyze the compiled .cpp file and its corresponding .h file.
- If IWYU suggests a change the build fails (-Xiwyu --error).
- To avoid false positives we use LLVM's libc++ as standard library
- We have a custom mappings file that adds some mappings that are
missing in IWYU's default mappings
We also had to add one IWYU pragma to prevent a false positive in
movegen.h.
uses a posix compatible script to find the native arch.
(based on ppigazzini's https://github.com/ppigazzini/stockfish-downloader )
use that native arch by default, no changes if ARCH is specified explicitly.
SF can now be compiled in an optimal way simply using
Now that qsearch has its own repetition detection we can flip the order of lines and remove the guard of depth < 0 which is not needed after reordering (i.e. it was there to prevent checking repetition again at depth ==0).
Cleanup code after dropping ICC support in favor of ICX
The commit removes all uses of ICC's __INTEL_COMPILER macro and other
references to ICC. It also adds ICX info to the compiler command and
fixes two typos in Makefile's help output.
Created by retraining the master net on a dataset composed by:
- adding Leela data from T60 jul-dec 2020, T77 nov 2021, T80 jun-jul 2023
- deduplicating and unminimizing parts of the dataset before interleaving
Trained initially with max epoch 800, then increased near the end of training
twice. First to 960, then 1200. After training, post-processing involved:
- greedy permuting L1 weights with https://github.com/official-stockfish/Stockfish/pull/4620
- greedy 2- and 3- cycle permuting with https://github.com/official-stockfish/Stockfish/pull/4640
In the list of datasets below, periods in the filename represent the sequence of
steps applied to arrive at the particular binpack. For example:
test77-dec2021-16tb7p.filter-v6-dd.min-mar2023.unminimized.binpack
1. test77 dec2021 data rescored with 16 TB of syzygy tablebases during data conversion
2. filtered with csv_filter_v6_dd.py - v6 filtering and deduplication in one step
3. minimized with the original mar2023 implementation of `minimize_binpack` in
the tools branch
4. unminimized by removing all positions with score == 32002 (`VALUE_NONE`)
Michael Chaly [Mon, 11 Sep 2023 12:37:18 +0000 (15:37 +0300)]
Do more futility pruning in qsearch
This patch introduces a third futility pruning heuristic in qsearch. The idea is
that the static exchange evaluation is much worse than the difference between
futility base and alpha. Thus we can assume that the probability of the move
being good enough to beat alpha is low so it can be pruned.
projects / stockfish / log