Other approach to fix issues with double comparisons
Summary: So it seems that my previous approach doesn't solve the issue completely, we are still seeing these errors, because std::numeric_limits<T>::epsilon() precision is not enough. We can do a different epsilon i.e 1e-6 but it will probably create other problems when values are very big or very small. I've read about how to do a proper comparison here: https://www.boost.org/doc/libs/1_66_0/libs/test/doc/html/boost_test/testing_tools/extended_comparison/floating_point/floating_points_comparison_theory.html but this seems to bee too complicated and slow to introduce relative error comparisons here. So I decided that we shouldn't do this comparison for floating types at all, because it doesn't make much sense, because we are trying to capture integer overflow with these guards. Example: "E0116 06:30:24.439344 3368082 Histogram.h:233] Called: lhs: 3.999999999999005240 rhs:4.000000000000000000 eps: 0.000000000000000222 diff: 0.000000000000994760" Reviewed By: yfeldblum Differential Revision: D13693535 fbshipit-source-id: e3e7dafa4517612c63bc8e22b62eeeb053677cb8
Showing
Please register or sign in to comment