Fix an hang on 32 bits while allocating big TT table

If size_t is defined as a 32 bit quanitity then we have an
overflow in the left term of the while condition if mbSize
is bigger then 2048.

For instance if mbSize is 2049 then when newSize will reach
0x80000000 (2048MB) comparison is still true, 'while' loops
again and we have an overflow in the expression (2*newSize)
so that result is 0 and at that point 'while' keeps looping
forever hanging the application.

This patch fixes the bug and also makes operator new do not
throw an exception upon failure but return a NULL pointer
instead.

No functional change.

Signed-off-by: Marco Costalba <mcostalba@gmail.com>

diff --git a/src/tt.cpp b/src/tt.cpp
index 8918e515fd5180c8c5a5f21523c4f5c2e2268538..20ac1608e2fee1eb0e1338c3dbbe940275cc9f36 100644 (file)
--- a/src/tt.cpp
+++ b/src/tt.cpp
@@ -55,18 +55,18 @@ void TranspositionTable::set_size(size_t mbSize) {
 
   size_t newSize = 1024;
 
-  // Transposition table consists of clusters and
-  // each cluster consists of ClusterSize number of TTEntries.
-  // Each non-empty entry contains information of exactly one position.
-  // newSize is the number of clusters we are going to allocate.
-  while ((2 * newSize) * sizeof(TTCluster) <= (mbSize << 20))
+  // Transposition table consists of clusters and each cluster consists
+  // of ClusterSize number of TTEntries. Each non-empty entry contains
+  // information of exactly one position and newSize is the number of
+  // clusters we are going to allocate.
+  while (2ULL * newSize * sizeof(TTCluster) <= (mbSize << 20))
       newSize *= 2;
 
   if (newSize != size)
   {
       size = newSize;
       delete [] entries;
-      entries = new TTCluster[size];
+      entries = new (std::nothrow) TTCluster[size];
       if (!entries)
       {
           std::cerr << "Failed to allocate " << mbSize

diff --git a/src/tt.h b/src/tt.h
index 94ee78e70f233f9126bee035e1213403551bee8f..c37b9d3ff6f74c96bbfe3853d8747f0f72804933 100644 (file)
--- a/src/tt.h
+++ b/src/tt.h
@@ -46,7 +46,7 @@ class SimpleHash {
 public:
   SimpleHash() {
 
-    entries = new Entry[HashSize];
+    entries = new (std::nothrow) Entry[HashSize];
     if (!entries)
     {
         std::cerr << "Failed to allocate " << HashSize * sizeof(Entry)