Provide WDL statistics

A number of engines, GUIs and tournaments start to report WDL estimates
along or instead of scores. This patch enables reporting of those stats
in a more or less standard way (http://www.talkchess.com/forum3/viewtopic.php?t=72140)

The model this reporting uses is based on data derived from a few million fishtest LTC games,
given a score and a game ply, a win rate is provided that matches rather closely,
especially in the intermediate range [0.05, 0.95] that data. Some data is shown at
https://github.com/glinscott/fishtest/wiki/UsefulData#win-loss-draw-statistics-of-ltc-games-on-fishtest
Making the conversion game ply dependent is important for a good fit, and is in line
with experience that a +1 score in the early midgame is more likely a win than in the late endgame.

Even when enabled, the printing of the info causes no significant overhead.

Passed STC:
LLR: 2.94 (-2.94,2.94) {-1.50,0.50}
Total: 197112 W: 37226 L: 37347 D: 122539
Ptnml(0-2): 2591, 21025, 51464, 20866, 2610
https://tests.stockfishchess.org/tests/view/5ef79ef4f993893290cc146b

closes https://github.com/official-stockfish/Stockfish/pull/2778

No functional change

diff --git a/Readme.md b/Readme.md
index 2b1de86be655775c6514257a05c6ea561c7c52b7..e60ac7185d7532a278e2b7856fe6e62cf6825fe6 100644 (file)
--- a/Readme.md
+++ b/Readme.md
@@ -66,6 +66,11 @@ Currently, Stockfish has the following UCI options:
     If enabled by UCI_LimitStrength, aim for an engine strength of the given Elo.
     This Elo rating has been calibrated at a time control of 60s+0.6s and anchored to CCRL 40/4.
 
     If enabled by UCI_LimitStrength, aim for an engine strength of the given Elo.
     This Elo rating has been calibrated at a time control of 60s+0.6s and anchored to CCRL 40/4.
 

+  * #### UCI_ShowWDL
+    If enabled, show approximate WDL statistics as part of the engine output.
+    These WDL numbers model expected game outcomes for a given evaluation and
+    game ply for engine self-play at fishtest LTC conditions (60+0.6s per game).
+

   * #### Move Overhead
     Assume a time delay of x ms due to network and GUI overheads. This is useful to
     avoid losses on time in those cases.
   * #### Move Overhead
     Assume a time delay of x ms due to network and GUI overheads. This is useful to
     avoid losses on time in those cases.

diff --git a/src/search.cpp b/src/search.cpp
index 0fa399882c287ead30231442edbc416af1b5aa94..f14bdf77055970a175f4e2c5c8b20ff8760da769 100644 (file)
--- a/src/search.cpp
+++ b/src/search.cpp
@@ -1835,6 +1835,9 @@ string UCI::pv(const Position& pos, Depth depth, Value alpha, Value beta) {
          << " multipv "  << i + 1
          << " score "    << UCI::value(v);
 
          << " multipv "  << i + 1
          << " score "    << UCI::value(v);
 

+      if (Options["UCI_ShowWDL"])
+          ss << UCI::wdl(v, pos.game_ply());
+

       if (!tb && i == pvIdx)
           ss << (v >= beta ? " lowerbound" : v <= alpha ? " upperbound" : "");
 
       if (!tb && i == pvIdx)
           ss << (v >= beta ? " lowerbound" : v <= alpha ? " upperbound" : "");
 

diff --git a/src/uci.cpp b/src/uci.cpp
index 11d5adc644ebf2ef3693d10c79e50f735bf4196c..bb57c80b5f6869fd4bd3c68fe225e7f7bce3555d 100644 (file)
--- a/src/uci.cpp
+++ b/src/uci.cpp
@@ -19,6 +19,7 @@
 */
 
 #include <cassert>
 */
 
 #include <cassert>

+#include <cmath>

 #include <iostream>
 #include <sstream>
 #include <string>
 #include <iostream>
 #include <sstream>
 #include <string>


@@ -182,6 +183,28 @@ namespace {
          << "\nNodes/second    : " << 1000 * nodes / elapsed << endl;
   }
 
          << "\nNodes/second    : " << 1000 * nodes / elapsed << endl;
   }
 

+  // The win rate model returns the probability (per mille) of winning given an eval
+  // and a game-ply. The model fits rather accurately the LTC fishtest statistics.
+  int win_rate_model(Value v, int ply) {
+
+     // The model captures only up to 240 plies, so limit input (and rescale)
+     double m = std::min(240, ply) / 64.0;
+
+     // Coefficients of a 3rd order polynomial fit based on fishtest data
+     // for two parameters needed to transform eval to the argument of a
+     // logistic function.
+     double as[] = {-8.24404295, 64.23892342, -95.73056462, 153.86478679};
+     double bs[] = {-3.37154371, 28.44489198, -56.67657741,  72.05858751};
+     double a = (((as[0] * m + as[1]) * m + as[2]) * m) + as[3];
+     double b = (((bs[0] * m + bs[1]) * m + bs[2]) * m) + bs[3];
+
+     // Transform eval to centipawns with limited range
+     double x = Utility::clamp(double(100 * v) / PawnValueEg, -1000.0, 1000.0);
+
+     // Return win rate in per mille (rounded to nearest)
+     return int(0.5 + 1000 / (1 + std::exp((a - x) / b)));
+  }
+

 } // namespace
 
 
 } // namespace
 
 


@@ -269,6 +292,22 @@ string UCI::value(Value v) {
 }
 
 
 }
 
 

+/// UCI::wdl() report WDL statistics given an evaluation and a game ply, based on
+/// data gathered for fishtest LTC games.
+
+string UCI::wdl(Value v, int ply) {
+
+  stringstream ss;
+
+  int wdl_w = win_rate_model( v, ply);
+  int wdl_l = win_rate_model(-v, ply);
+  int wdl_d = 1000 - wdl_w - wdl_l;
+  ss << " wdl " << wdl_w << " " << wdl_d << " " << wdl_l;
+
+  return ss.str();
+}
+
+

 /// UCI::square() converts a Square to a string in algebraic notation (g1, a7, etc.)
 
 std::string UCI::square(Square s) {
 /// UCI::square() converts a Square to a string in algebraic notation (g1, a7, etc.)
 
 std::string UCI::square(Square s) {

diff --git a/src/uci.h b/src/uci.h
index b845889bcc158779dbaad997c0451adcaf60b00e..ad954d9f7128f013ec1e8caa05987fbc60936e94 100644 (file)
--- a/src/uci.h
+++ b/src/uci.h
@@ -73,6 +73,7 @@ std::string value(Value v);
 std::string square(Square s);
 std::string move(Move m, bool chess960);
 std::string pv(const Position& pos, Depth depth, Value alpha, Value beta);
 std::string square(Square s);
 std::string move(Move m, bool chess960);
 std::string pv(const Position& pos, Depth depth, Value alpha, Value beta);

+std::string wdl(Value v, int ply);

 Move to_move(const Position& pos, std::string& str);
 
 } // namespace UCI
 Move to_move(const Position& pos, std::string& str);
 
 } // namespace UCI

diff --git a/src/ucioption.cpp b/src/ucioption.cpp
index c268c9755f9eed39ea9d132f112d9dfd57af2a74..4befa6ac7c29ad8785655fa3838aca0bd032846c 100644 (file)
--- a/src/ucioption.cpp
+++ b/src/ucioption.cpp
@@ -74,6 +74,7 @@ void init(OptionsMap& o) {
   o["UCI_AnalyseMode"]       << Option(false);
   o["UCI_LimitStrength"]     << Option(false);
   o["UCI_Elo"]               << Option(1350, 1350, 2850);
   o["UCI_AnalyseMode"]       << Option(false);
   o["UCI_LimitStrength"]     << Option(false);
   o["UCI_Elo"]               << Option(1350, 1350, 2850);

+  o["UCI_ShowWDL"]           << Option(true);

   o["SyzygyPath"]            << Option("<empty>", on_tb_path);
   o["SyzygyProbeDepth"]      << Option(1, 1, 100);
   o["Syzygy50MoveRule"]      << Option(true);
   o["SyzygyPath"]            << Option("<empty>", on_tb_path);
   o["SyzygyProbeDepth"]      << Option(1, 1, 100);
   o["Syzygy50MoveRule"]      << Option(true);