]> git.sesse.net Git - movit/commit
Fix a bug where combined fp16 weights would be horribly wrong.
authorSteinar H. Gunderson <sgunderson@bigfoot.com>
Wed, 23 Sep 2015 23:59:47 +0000 (01:59 +0200)
committerSteinar H. Gunderson <sgunderson@bigfoot.com>
Thu, 24 Sep 2015 00:04:33 +0000 (02:04 +0200)
commit6c954b4f0bff0743e13ce6ddcee8bda15b3af234
tree0aac2be5201ad6d3e27b17cfa654fffa91de1b8d
parent645c0e312389d15c6661c152ce9635183e9ca307
Fix a bug where combined fp16 weights would be horribly wrong.

Seemingly weights were always returned as float, and then cast
to fp16_int_t -- without proper conversion! And sum_sq_error
would be calculated based on the correct value, not the broken-
casted one.

It's a small miracle the unit tests didn't catch this; they didn't
until I started introducing small errors for another reason.
Most real-world testing seems to have hit fp32, and thus this
wasn't caught there either.

Also make fp16_int_t a struct so that it is not implicitly
convertible to/from numeric types, so this never ever can happen again.
fp16.cpp
fp16.h
fp16_test.cpp
resample_effect.cpp
util.cpp