7

In the book Low-Level Programming: C, Assembly, and Program Execution on Intel® 64 Architecture it says,

On system call arguments The arguments for system calls are stored in a different set of registers than those for functions. The fourth argument is stored in r10 , while a function accepts the fourth argument in rcx!

The reason is that syscall instruction implicitly uses rcx. System calls cannot accept more than six arguments.

You can see this also mentioned in this Stack Overflow post,

A system-call is done via the syscall instruction. This clobbers %rcx and %r11, as well as %rax, but other registers are preserved.

I understand clobbering rax to store the return code, but why is rcx, and r11 clobbered in syscall? Is there a list of the specific syscalls that clobber rcx/r11? Is there a convention for the clobbering? Are they assumed safe in any syscalls?

Peter Cordes
  • 328,167
  • 45
  • 605
  • 847
Evan Carroll
  • 78,363
  • 46
  • 261
  • 468

1 Answers1

7

The syscall instruction uses rcx to store the address of the next instruction to return to, and r11 to save the value of the rflags register. These values will then be restored by the sysret instruction.

This is done by the CPU when executing the CPU instruction, so any OS-specific calling conventions need to avoid using these registers to pass arguments to syscalls.

interjay
  • 107,303
  • 21
  • 270
  • 254
  • 1
    What's the purpose of saving the value of rflags in `r11`, rather than just leaving it there? – Evan Carroll May 28 '18 at 17:50
  • @EvanCarroll It will be restored to rflags when the call returns. I think it's done in order to restore some of the more important flags which may be changed by the kernel code. – interjay May 28 '18 at 17:58
  • 1
    @EvanCarroll `sysret` also masks (ANDs) `rflags` before handing control to the OS. That'd be impossible without first saving `rflags`. You can take a look at the [`syscall`](https://www.felixcloutier.com/x86/SYSCALL.html) manual entry for more information. – Margaret Bloom May 28 '18 at 17:59