These SANE operations can sometimes return incorrect values for certain negative integers such as -2147483648 and -53021371269120 (numbers with at least 16 low-order zero bits in their two's-complement representation). To work around this, we now avoid calling FX2C or FX2L on negative numbers, generally by saving and restoring the sign separately.
These workarounds are used in several of the new <math.h> rounding functions, and also for code that converts floating-point values to comp or long long. There are some places in SysFloat that should be patched similarly, so we may still hit this problem in certain situations until that is done.
For conversion to signed long long, we can almost just use SANE's FX2C call to convert it to comp. But LLONG_MIN corresponds to the bit pattern of the comp NaN, so it has to be treated specially. (You might think SANE would return the comp NaN when converting out-of-range values, which would give the value we want, but somewhat strangely it does not.)
For conversion to unsigned long long, we check if the value is greater than LLONG_MAX (which is also the max comp value). If so, we subtract LLONG_MAX+1 from the floating-point value before conversion and add it back in to the integer afterward.
This is almost the same as ~CMP8 in SysLib, except that it does not use a static location to save the DP. This makes it slightly smaller/faster, and also makes it thread/interrupt safe.
The new m16.int64 file contains a new "negate8" macro (for 64-bit negation), as well as an improved version of "ph8". These have been added to the individual *.macros files, but if those are regenerated, m16.int64 should be used as an input to macgen ahead of m16.ORCA (which contains the original version of ph8).
SANE does not have built in operations for this, but all the values are exactly representable in the SANE extended format, so we just do the conversion ourselves based on the definition of the extended format.
This is a fairly straightforward adaptation of the strtol/strtoul code. The multiplication routine is modified to return the next 16 bits in X so that strtoull can detect overflows.
The ~CDIV8 routine is the same as ~DIV8 in SysLib, except that it negates the remainder if the numerator is negative. This complies with the definition of the % operator in C, which gives negative (or 0) remainders if the numerator is negative, such that (a/b)*b + a%b equals a.
The ~UDIV8 routine is an unsigned version, using the same core division loop but without the sign handling.
This file will be expanded with more support routines as needed.
The unsigned multiply routine uses the core multiplication loop from the 64-bit signed multiply routine in SysLib, with new entry/exit code for the unsigned version.
Stephen Heumann