Linux kernel function returns 1 instead of EINVAL-CodePudding

I'm trying to add a new system call to linux kernel:

asmlinkage long sys_set_status(int status) {
    if ((status != 0) && (status != 1))
        return -EINVAL; //-22
    current->status = status;
    return 0;
}

in syscall_64.tbl it is declared:

334 common  set_status      sys_set_status

in syscalls.h it is declared:

asmlinkage long sys_set_status(int status);

but when i test the return value:

int set_status(int status) {
    long r = syscall(334, status);
    return (int)r;
}

int main() {
    int res = set_status(-1); //illegal status, should return -EINVAL whish is -22
    cout << "return value is: " << res << endl;
    return 0;
}

i get:

return value is: -1

CodePudding user response：

long r = syscall(334, status);

From man syscall:

The return value is defined by the system call being invoked. In general, a 0 return value indicates success. A -1 return value indicates an error, and an error number is stored in errno.

You are not calling the system call directly, you are calling it via the libc syscall wrapper, which performs approximately this:

int syscall(num, ...)
{
  /* architecture-specific code to put system call number and args into
     appropriate registers */
  /* arch-specific code to execute the system call instruction */
  int rc = /* arch-specific code to get the result of the system call */ ;
  if (rc < 0) { errno = -rc; return -1; }
  return 0;
}

If you don't want this translation to happen, you would have to perform the architecture-specific parts yourself (in assembly), and then you would have the actual system call return value.

CodePudding user response：

As pointed out in Employed Russian's answer, negative return values from system calls get converted to a return value of -1 by the system call wrapper functions in libc and errno is set. So the return value of -1 is expected and the value of errno should be checked.

It is likely that errno will be set to ENOSYS rather than EINVAL because the kernel syscall code has been written in the old format that does work on most 64-bit kernels. The kernel code for the system call should be updated to use the SYSCALL_DEFINE1 wrapper macro (because the syscall has 1 parameter) as shown below:

#include <linux/syscalls.h>

SYSCALL_DEFINE1(set_status, int, status)
{
    if ((status != 0) && (status != 1))
        return -EINVAL; //-22
    current->status = status;
    return 0;
}