I decided to write a blog entry about my porting efforts of sydbox to IA64.
Thanks to ahf who gave me access to an IA64 box, I started porting sydbox to IA64.

All ptrace() related stuff of Sydbox resides in trace.c so that’s the file we’re interested in.
For x86 and x86_64 architectures everything was so easy,
just fill in some registry values and it’s all done. For IA64, however,
additional hackery is needed.

First problem, you can’t include both sys/ptrace.h and linux/ptrace.h due to
redefinition errors. This is easy to solve, I added an autoconf check:

    AC_CHECK_TYPES([struct pt_all_user_regs, struct ia64_fpreg],,,[#include <sys/ptrace.h>])

This one exports two defines to config.h namely HAVE_STRUCT_IA64_FPREG and
HAVE_STRUCT_PT_ALL_USER_REGS , using this two defines it’s possible to include
both files using some preprocessor hacks:

    #include <sys/ptrace.h>
    /* We need additional hackery on IA64 to include linux/ptrace.h to avoid
     * redefinition errors.
     */
    #if defined(IA64)
    #ifdef HAVE_STRUCT_IA64_FPREG
    #define ia64_fpreg XXX_ia64_fpreg
    #endif // HAVE_STRUCT_IA64_FPREG
    #ifdef HAVE_STRUCT_PT_ALL_USER_REGS
    #define pt_all_user_regs XXX_pt_all_user_regs
    #endif // HAVE_STRUCT_PT_ALL_USER_REGS
    #include <linux/ptrace.h>
    #undef ia64_fpreg
    #undef pt_all_user_regs
    #endif // defined(IA64)

and problem solved.

The second problem is figuring out how to get system call number, this is easy too.
PT_R15 is the registry we need.

The third problem is getting and setting the return value of system calls. On
IA64 the registry PT_R10 has the return value of the system call. One last thing
is the errno. The errno should reside in the PT_R8 registry and it should be
positive unlike x86 and x86_64 where return value of the system call is the
negated errno.

So far so good, the last problem is getting and setting arguments of functions.
This is a bit too long to explain, interested readers are encouraged to look at
the trace_ia64_peek function in trace.c.

edit: Highlight code.