I want to implement a lock free counter, a 4-byte int, in System V shared memory. The writer is a C program, the reader is a Python program. Working roughly like this:

• C code updates counter in an atomic operation
• Python code reads counter and has a consistent view of memory (eventual consistency is perfectly acceptable)
• No locks are implemented to achieve this

Within the C language there are atomic get/update operations that allow for this and guarantee memory consistency, I believe the same is true in Python.

However, as I understand it, the assumptions in C regarding atomic operations do not necessarily apply to code written and compiled in another language and compiler.

Is there a way to achieve a consistent view of shared memory across languages that doesn't involve implementing low level locks?

CodePudding user response：

Is there a way to achieve a consistent view of shared memory across languages that doesn't involve implementing low level locks?

No, not in general.

First I would say this has nothing to do with languages but more with the actual platforms, architecture, implementation or operating system.

Because languages differ quite strongly, take Python for example: It has no language native way of accessing memory directly or lets say in an low level manner. However some of its implementations do offer its own API. Languages intended for such low level use have abstractions for that, as C, C or Rust have. But that abstractions are then implemented often quite differently, as they often depend on where the code is run, interpreted or compiled for. An integer for some architectures are big endian, on most, like x86 or arm, its little endian. Operating systems also have a say, as for example memory is used and abstracted.

And while many languages have a common abstractions of linear memory, its gets even messier with atomics: The compiler for C could generate machine code i.e. assembly that check if the CPU run on does support new fancy atomic integer instructions and use them or fall back on often supported atomic flags plus the integer. It could just rely on operating system, a spin lock, or standardized APIs defined by POSIX, SystemV, Linux, Windows if the code has the luxury of running in an operating system managed environment .

For non imperative languages it gets even messier.

So in order to exchange data between languages, the implementations of those languages have to use some common exchange. They could try to do that directly via shared memory for instance. This is then called an Application Binary Interface (ABI), as a memory abstraction is at least common to them a priori. Or the operating system or architecture might even standardized such things or even supports APIs.

System V would be an API designed for such interchange but since AFAIK it does not have an abstraction for atomics or lock less abstractions, the answer stays no, even with the System V context out of the title.

CodePudding user response：

I'm going to take issue with some of Superlokkus' assertions.

The mmap primitive is available in both C and Python. That primitive gives you memory that is in a physical page shared by both processes. Different virtual addresses, same physical page of memory. Changes by one process are immediately viewable by the other process. That's how it has to work. It operates at a hardware level. Abstractions are irrelevant.

Now, that DOESN'T mean you can get notification of those changes. If you are polling in a loop (presumably a friendly loop with sleeping in between checks), then you will see the change the next time you check.