Home > Net >  Python ctypes: when do you need to manually add b`\0` to a bytes object?
Python ctypes: when do you need to manually add b`\0` to a bytes object?

Time:12-27

In Python ctypes, when, if ever, do you need to manually add the null/zero b'\0' terminator when passing bytes to a function that expects null terminated data?

Specifically for the 3 cases (but others welcome)

  • If the function parameter has been declared with c_char_p via its argtypes

  • If the function has not had its parameter declared via argtypes

  • Using memmove, if the interface expects a null terminated string at a memory address,

    memmove(target, my_string.encode()   b'\0', len(my_string.encode())   1)
    

    or can you do

    memmove(target, my_string.encode(), len(my_string.encode())   1)
    

Context: I add b'\0' out of paranoia in sqlite-s3-query, and trying to work out if I can remove them. It seems to work fine if I do remove them, but I realise that there could just happen to be null bytes in the right places so everything just works. I'm looking for a stronger guarantee that the null bytes are there by design.

CodePudding user response:

At least in CPython, the internal buffer for a bytes object is always null-terminated and there is no need to add another one. Whether you specify .argtypes or not, the pointer generated will point to this buffer.

Ref: https://docs.python.org/3/c-api/bytes.html#c.PyBytes_AsString:

char *PyBytes_AsString(PyObject *o)
Part of the Stable ABI.
Return a pointer to the contents of o. The pointer refers to the internal buffer of o, which consists of len(o) 1 bytes. The last byte in the buffer is always null, regardless of whether there are any other null bytes....

  • Related