15

I'm self-studying how compilers works. I'm learning by reading the disassembly of GCC generated code from small 64-bit Linux programs.

I wrote this C program:

#include <stdio.h> int main() { for(int i=0;i<10;i++){ int k=0; } }

After using objdump I get:

00000000004004d6 <main>: 4004d6: 55 push rbp 4004d7: 48 89 e5 mov rbp,rsp 4004da: c7 45 f8 00 00 00 00 mov DWORD PTR [rbp-0x8],0x0 4004e1: eb 0b jmp 4004ee <main+0x18> 4004e3: c7 45 fc 00 00 00 00 mov DWORD PTR [rbp-0x4],0x0 4004ea: 83 45 f8 01 add DWORD PTR [rbp-0x8],0x1 4004ee: 83 7d f8 09 cmp DWORD PTR [rbp-0x8],0x9 4004f2: 7e ef jle 4004e3 <main+0xd> 4004f4: b8 00 00 00 00 mov eax,0x0 4004f9: 5d pop rbp 4004fa: c3 ret 4004fb: 0f 1f 44 00 00 nop DWORD PTR [rax+rax*1+0x0]

Now I have some doubts.

  1. What is that NOP at the end for, and why is it there? (alignment?)

  2. I'm compiling with gcc -Wall <program.c>. Why am I not getting the warning control reaches end of non-void function?

  3. Why doesn't the compiler allocate space on the stack with sub rsp,0x10? Why doesn't it use the rbp register for referencing local stack data?

    PS: If I call a function (like printf) in the for loop, why does the compiler suddenly generate sub rsp,0x10? Why does it still references local data with the rsp register. I expect the generated code to reference local stack data with rbp!

Improve this question
12
  • 11
    a little tip: if you want to play around with compilers check out : godbolt.org Commented Mar 24, 2017 at 8:36
  • 5
    None of the answers really captures why #3 occurs. You will find this in Linux 64-bit code. The compiler is taking advantage of the Red Zone. You can find information of this red zone in the 64-bit System V Linux ABI . In that copy of the document review section 3.2.2: Stack Frame .Since your function is a leaf function (doesn't call other functions) it can take advantage of the fact that 128 bytes below the current RSP won't be clobbered by signal handling etc. Commented Mar 24, 2017 at 9:46
  • 2
    Because the 128 bytes below the current value of RSP (in leaf functions) are safe there is no need to adjust RSP if the functions stack based data can fit in that 128 bytes. That is why you don't see RSP adjusted here. One observation. The Red Zone doesn't exist in 32-bit Linux, so compiling as 32-bit code you should see the behaviour you might have expected. Commented Mar 24, 2017 at 9:58
  • 1
    As for the question as to why when you add the test function the compiler subtract 16 bytes although less is needed is related to 64-bit Linux ABI requirement that at the point of any function call (like calling test) the stack needs to be 16 (and possibly 32-byte aligned). Since the stack is aligned at the point of the call, the CALL itself pushes 8 byte return address. That misaligns the stack by 8. The push rbp subtracts another 8 which makes it 16 byte aligned again. Now you need local variable data. Compilers usually allocate enough bytes to maintain the 16-byte alignment Commented Mar 24, 2017 at 10:54
  • 1
    By subtracting 16-bytes, by the time it reaches the call to test the stack remains 16-byte aligned and all is good.. Commented Mar 24, 2017 at 10:55

3 Answers 3

Reset to default
12

Regarding the second question, since the C99 standard it's allowed to not have an explicit return 0 in the main function, the compiler will add it implicitly. Note that this is only for the main function, no other function.

As for the third question, the rbp register acts as the frame pointer.

Lastly the PS. It's likely that the called function is using 16 bytes (0x10) for the arguments passed to the function. The subtraction is what "removes" those variables from the stack. Could it possibly be two pointers you pass as arguments?

If you're serious learning how compilers in general works, and possibly want to create your own (it's fun! :)), then I suggest you invest in some books about the theory and practice of it. The dragon book is an excellent addition to any programmers bookshelf.

Improve this answer
Sign up to request clarification or add additional context in comments.

12 Comments

plus 1/3 in integer arithmetic
@Bathsheba I didn't even expected that much! :)
Dunnit. Perhaps "why can't we have a rational type for the number of votes" is a meta question?
Because we don't want the upvotes to go from 3 to 2.99999999997
|
6

  1. Yes, the nop is for alignment. Compilers use different instructions for different lengths of padding needed, knowing that modern CPU will be pre-fetching and decoding several instructions ahead.

  2. As others have said, the C99 standard returns 0 from main() by default if there's no explicit return statement (see 5.1.2.2.3 in C99 TC3), so no warning is raised.

  3. The 64-bit System V Linux ABI reserves a 128-byte "red zone" below the current stack pointer that leaf functions (functions that do not call any other functions - and your main() is one such) can use for local variables and other scratch values without having to sub rsp / add rsp. And so rbp == rsp.

And for the PS: when you call a function in the for() loop (or anywhere in your main()), main() is no longer a leaf function, so the compiler can no longer use the red zone. That's why the it allocates space on the stack with sub rsp, 0x10. However, it knows the relationship between rsp and rbp, so it can use either when accessing data.

Improve this answer

Comments

6

Anything after the ret cannot be relied on to be code. Decoding as nop means "No OPeration"

The 2nd point is the compiler detecting you leave the main function without returning a value and it inserts a return 0 (only defined for main).

The rbp register, with bp meaning "Base Pointer", points to the stack frame of the currect function. A function call often results in the function entry saving rbp and using the current value of rsp for rbp. Fetching/storing function arguments and local variables are done relative to rbp.

I think your third question needs some more attention, "Why doesn't the compiler allocate space on the stack with sub rsp,0x10? Why doesn't it use the rbp register for referencing local stack data?"

Actually, the compiler does allocate space on the stack. But it does not change the stackpointer. It can do that because the functon calls no other functions. It just uses space below the curent sp (the stack grows down) and it uses rbp to access i ([rbp-0x8]) and k ([rbp-0x4]).

I must add the following note: not adjusting sp for the use of local variables seems not interrupt safe and so the compiler relies on the hardware automatically switching to a system stack when interrupts occur. Otherwise, the first interrupt that came along would push the instruction pointer onto the stack and would overwrite the local variable.

Question of interrupts solved in Compiler using local variables without adjusting RSP

Improve this answer

5 Comments

After the ret call, then the compiler just throw random opcode?
@Ofey : What hasn't been captured by any of the answers here is the why that NOP exists. In most cases when it is after a function (after the ret in this case), it has been placed there so that the next function starts on a 16-byte boundary. Take the address at the start of the NOP (0x4004fb in your objdump) and add the 5 bytes the NOP takes. You get 0x400500. 0x400500 is evenly divisible by 16.This is done for performance reasons (related to cache lines).Likely in your objdump you have a function that appears right after main.You'll see NOPs in a function to align loops like this as well.
@MichaelPetch Thank you! Usually when I see nop I always think about alignment, in this case I wasn't able to understand where to align. Thank you again!
I also used nops to patch an executable.
@PaulOgilvie Me also,during some crackme challenges.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.