Stingray (5.7.0.0)

What's Changed

• Decrease learning rate and drop learning rate during Neural Network Training. by @TheBlackPlague in #87
• Decrement History Heuristic Scores for non-best moves. by @TheBlackPlague in #88

NNUE:

       IN          ACCUMULATOR                               HIDDEN                                    OUT
 ______________      _______      ______________________________________________________________      _____
| WHITE: (768) | -> | (256) | -> | ClippedReLU(0, 1) -> (256) \                                 |    |     |
|              |    |       |    |                             \                                |    |     |
|              |    |       |    |                              CONCATENATE(ColorToMove): (512) | -> | (1) |
|              |    |       |    |                             /                                |    |     |
| BLACK: (768) | -> | (256) | -> | ClippedReLU(0, 1) -> (256) /                                 |    |     |
 --------------      -------      --------------------------------------------------------------      -----

TAG: EXP-0006
Data: 1.7B FEN --- DEPTH: 9 (1.2B) + NODES: 5K (500M)
Batch Size: 500000
Epochs: 80
LR: 5e-3
LR Drop: 30
WDL: 0.4

Full Changelog: 5.6.0.0...5.7.0.0

ELO Difference

Using UHO_XXL_+0.90_+1.19.epd:

TC: 10s + 0.1s (STC)

ELO   | 73.55 +- 5.43 (95%)
CONF  | 10.0+0.10s Threads=1 Hash=16MB
GAMES | N: 10000 W: 4156 L: 2070 D: 3774

TC: 60s + 0.6s (LTC)

ELO   | 70.94 +- 5.12 (95%)
CONF  | 60.0+0.60s Threads=1 Hash=256MB
GAMES | N: 10000 W: 3802 L: 1788 D: 4410