Questions tagged [number-formats]
For questions about method for storing and manipulating numbers on computer systems, such as floating point or binary representations.
91 questions
0 votes
1 answer
53 views
What numeral system does a computer use?
We say that a computer calculates in binary via logic gates. A human in everyday calculates in base-10 numeral system. A number can be represented in 7 lines that form an "8" as calculators ...
- 103
0 votes
1 answer
161 views
How to check if a real number, in a decimal form, has an accurate IEEE 754 representation?
Perhaps the answer is simple, but I am looking for ideas. I need to generate a set of a minimum of 100-200 real numbers, all having an accurate computer representation (according to the IEEE 754 ...
- 1
1 vote
1 answer
76 views
Unsigned-to-two's complement algorithm
This question is inspired by Equation 2.8 in Bryant & O'Hallaron's Computer Systems which says: $B2U_w (\vec{x})$ (binary-to-unsigned) and $B2T_w (\vec{x})$ (binary-to-two's complement) can be ...
- 117
2 votes
1 answer
528 views
Data type implementation of 1.58 bits
In Large Language Models, using 1-bit binary weights (BitNet: Scaling 1-bit Transformers for Large Language Models - Wang et al, 2023) instead of 32-bit floating point weights has numerous advantages. ...
- 123
1 vote
2 answers
699 views
Convert a rational number to a floating-point number exactly
We have two integers, $n$ and $d$. They are coprime (the only positive integer that is a divisor of both of them is $1$). They may be implemented as something that fits in a machine register, or they ...
- 201
1 vote
2 answers
313 views
How does Decimal to Binary conversion algorithm work?
I'm reading a book which says that to transform a number from decimal representation to binary representation you have to follow these steps: Take the integer part (the one before the point) of the ...
- 315
1 vote
2 answers
220 views
Uses of Hex and Octal numbers in programming
Where are these number patterns, Octal and Hex, being used? I read this but I want to know where they are used in programming.
- 111
1 vote
1 answer
252 views
What does it mean unambiguously that a number is value 0 up to numerical precision?
I was reading that a quantity $x$ is $0$ upt to numerical precision. What does this statement formally mean -- especially in the context of numerical methods or real computers. I looked up in google ...
- 3,124
0 votes
1 answer
127 views
IEEE 754 conversion
I'm trying to convert 3.2 into IEEE 754 format. We find that $(3)_2=11$ and we also find that $0.2*2=0.4 -0$ $0.4*2=0.8 -0$ $0.8*2=1.6 -1$ $0.6*2=1.2 -1$ and this cycle repeats so $.2=00110011...$ ...
- 135
4 votes
0 answers
198 views
Uniformly random decimal numbers
Due to finite precision of number representations, we face situations like: In: 0.1+0.1+0.1==0.3 Out: False (on my ...
- 685
1 vote
1 answer
457 views
Prove every number in double precision 32-bit floating-point format can be represented in 64-bit format
Theorem: Prove every number in double precision 32-bit floating-point format can be represented in double precision 64-bit floating point-format. 64-bit format: Attempt: Let $ b = b_0 ,...,b_{31} $ ...
- 243
1 vote
1 answer
80 views
Is this a good way to arrange data in a tree?
I was recently learning about Binary Search Trees(BSTs) and thought it could be made even more efficient by making some changes. As binary search trees have numbers greater than the root node on the ...
- 139
1 vote
3 answers
6k views
Negative Numbers in 32 bit Floating Point IEEE Numbers
So I understand the logic behind converting positive decimal numbers to IEEE 32 bit floating numbers but I'm not completely sure behind the negative one's. If for example we have a decimal number say -...
- 13
6 votes
3 answers
5k views
Why is 2s complement of 000 equal to 111, but 9s complement of 000 is not 888?
I'm pretty confused so I hope I don't mix up the different terms here. The two's complement representation of decimal 0 is simply ...
- 231
1 vote
1 answer
3k views
Arithmetic on signed 12-bit octal number stored in sign magnitude form
What is 4365 − 3412 when these values represent signed 12-bit octal numbers stored in sign-magnitude format? The result should be written in octal. Show your work. Octal to binary: 4365: 100 011 110 ...
- 147