Python efficient assigning of even and odd bits to a bitstring/bit

I receive serialized DDR data from a setup with 8 chips. Due to the way the readout is implemented in hardware, the data that is received by the computer has the following structure:

bits 0 and 1 of Chip A, bits 0 and 1 of Chip B, ... bits 2 and 3 of Chip A, bits 2 and 3 of Chip B, ...

In order to make sense of the individual reply of each chip, the data needs to be split:

import bitstring

data = bitstring.BitArray(1024)  # contains serialized DDR data of 8 chips (here just 0 values for demo purposes)

even_bits_of_chip_A = data[0::16]  # starting at position 0, every 16th bit is an even bit of chip A
odd_bits_of_chip_A = data[1::16]  # starting at position 1, every 16th bit is an odd bit of chip A

data_of_chip_A = bitstring.BitArray(len(even_bits_of_chip_A)   len(odd_bits_of_chip_A))
data_of_chip_A[0::2] = even_bits_of_chip_A
data_of_chip_A[1::2] = odd_bits_of_chip_A

This code works fine and does what I want it to do. However, it is not that fast (considering I have to do this for all 8 chips, and generally with a lot more than 1024 bits). Is there a way to speed it up?

Of course the code can be rewritten like this:

import bitstring

data = bitstring.BitArray(1024)

data_of_chip_A = bitstring.BitArray(int(len(data) / 8))
data_of_chip_A[0::2] = data[0::16]
data_of_chip_A[1::2] = data[1::16]

This avoids the creation of the even_bits, odd_bits variables and increases performance. But the final step of assigning values to every second bit of data_of_chip_A still takes quite some time. Is there a way to for example join two bit_arrays in "alternating" fashion?

CodePudding user response：

The issue might be that the stepping assignments are going to be quite slow as the BitArray is having to be manipulated a lot to remain as a dense object representing the data. It could be faster to instead go via a string as an intermediary:

import bitstring

data = bitstring.BitArray(1024)
data_bin = data.bin  # Convert to ordinary str of '0' and '1's
# Create a list to contain single '0' or '1' characters
data_of_chip_A_bin = [''] * (len(data) // 8)
data_of_chip_A_bin[0::2] = data_bin[0::16]
data_of_chip_A_bin[1::2] = data_bin[1::16]
# Convert back to dense binary object
data_of_chip_A = bitstring.BitArray(bin=''.join(data_of_chip_A_bin))

...or that might be slower. Hard to tell without more work. In general the bitarray module mentioned in the comments is going to be faster doing the bit manipulation work in the original answer as it's a compiled C module as opposed to working in pure Python.

CodePudding user response：

As suggested in one of the comments, I switched to bitarray. It works fairly similar to bitstring and is way faster. Together with some simple multiprocessing the script runs 24x faster now. Thanks for the suggestions!

from bitarray import bitarray

data = bitarray(1024)  # empty/random array with 1024 bits
even_bits_of_chip_A = data[0::16]  # starting at position 0, every 16th bit is an even bit of chip A
odd_bits_of_chip_A = data[1::16]  # starting at position 1, every 16th bit is an odd bit of chip A

data_of_chip_A = bitarray(len(even_bits_of_chip_A)   len(odd_bits_of_chip_A))
data_of_chip_A[0::2] = even_bits_of_chip_A
data_of_chip_A[1::2] = odd_bits_of_chip_A