Questions tagged [arithmetic]
The arithmetic tag has no summary.
30 questions
-4 votes
1 answer
196 views
if I write 16 bit floating point number representation of 2044 and 2045.5 it's seems like the same, which is 0 1001001 11111111 why is that? [closed]
floating point representations## exponent == 7 bits ,mantissa == 8 bits 2044 = 0 1001001 11111111 2045.5 = 0 1001001 11111111 why is this happening?? what is the error? Is anything wrong with the ...
4 votes
1 answer
5k views
Semantics of simulating a 64bit integer with two 32bit integers (hi, lo)
I am working in a system which only supports 32bit integers, in order to have a 64bit (unsigned) integer I decided to simply use two 32bit integers with one being the upper 32 bits (hi), and the other ...
- 143
-2 votes
2 answers
5k views
Why does Integer.MIN_VALUE - Integer.MAX_VALUE equals to 1 in Java?
If you run the statement: System .out.println( (Integer.MIN_VALUE)-(Integer .MAX_VALUE )); In java it returns 1 as the answer. Is this because Java just considers those values as 32bit integers ...
- 289
-1 votes
2 answers
167 views
Describe a slope (N/M), approximately as small number of fractions (n/m)
What algorithm can I use to describe a specified gradient (N/M) approximately as the sum of a set of rational fractions { (n1/m1) + (n2/m2) … } ? Design constraints: The algorithm takes as input (N, ...
- 188
2 votes
5 answers
498 views
Shortening a boolean AND with third operand
I'm trying to calculate the sum of 2 bits using basic binary arithmetic and currently, I'm doing this: function Add(bool a, bool b, bool carry) { return { Result: a ^ b ^ carry, ...
- 31
3 votes
1 answer
4k views
Difficulties with Two's complement in Assembly
Everything discussed here will refer to Intel 8085 (8-bit architecture). When using two's complement number "conversion" : 1.) If we take SBI 0F (subtraction) for example; immediate value 0F(hex) or ...
- 293
4 votes
4 answers
3k views
Why are arbitrary-precision decimals used over rational numbers in practice?
The basic arithmetic a computer chip can do only works on numbers(integers or floating point) of a fixed size. There are many algorithms that could be extended to work on numbers of arbitrary size (...
- 4,947
0 votes
1 answer
468 views
Combine sequences of numbers with "variable bitlengths" into short unique strings
It is not unlikely that what I want to do is not possible, but it doesn't hurt to ask. Imagine a set of lists, each containing positive integers (in my case, a list always consists of four integers, ...
- 3