Effeciently check if a list is full of zeros-CodePudding

I have a json packet with three channels coming in every second. Im doing filtering on these channels, but usally 2 of them are empty(just filled with zeros) and there is only one channel that are filled with values. The problem is that you can change which channel that transmits the data on an app. So in the beginning channel 1 can be filled with data, but then it can change to channel 2, then I also want to switch to filter channel 2 instead of channel 1. This has to happen efficently. These are some of the solutions I have tried now

class manage:
    self.channel_number = 1
    if np.any(json[self.channel_number]):
       self.channel_number = next_channel

    filtered = filter_signal(json[self.channel_number])

another method I tried is

just checking the max value (as this should never be 0) of the list and then do the same as above

class manage:
    self.channel_number = 1
    if max(json[self.channel_number]) == 0.0:
       self.channel_number = next_channel

    filtered = filter_signal(json[self.channel_number])

Now the checking for the max value is the fastest of the two. But doesnt seem effective to have this check every second as it is not very common to switch channel midway

the json data is setup like this: json['data'] is a list of all the channels, so getting channel 1 would be json['data'][0] etc. The size of the data is around 200-400 and is on a list format. Therefor will max be faster then np.max Any tips to for the check if the channels is 0 or filled?

edit: Example of list: [0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.05, 0.05, 0.05, 0.05, 0.075, 0.075, 0.075, 0.075, 0.1, 0.125, 0.125, 0.125, 0.15, 0.15, 0.15, 0.15, 0.175, 0.2, 0.175, 0.175, 0.175, 0.175, 0.175, 0.175, 0.15, 0.125, 0.125, 0.1, 0.075, 0.075, 0.05, 0.025, 0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, -0.025, 0.0, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, 0.0, 0.025, 0.1, 0.15, 0.175, 0.075, 0.05, 0.15, 0.35, -0.375, -0.325, -0.275, -0.2, -0.15, -0.1, -0.05, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

example empty list: [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]

CodePudding user response：

For analyzing a single already parsed list, simply using any might be best. Benchmarks with your full/empty sample lists:

 9061 ns  np.any(full)
 3468 ns  max(full)
  427 ns  any(x != 0 for x in full)
   80 ns  any(full)

 9019 ns  np.any(empty)
 3604 ns  max(empty)
14775 ns  any(x != 0 for x in empty)
 1668 ns  any(empty)

But like I said, maybe you could check the JSON already and then only parse the relevant part/channel. Benchmarks for some ideas:

   31 ns  '5' in full_json
   29 ns  full_json == empty_json_for_comparison
   82 ns  len(full_json) == empty_json_length

   41 ns  '5' in empty_json
   50 ns  empty_json == empty_json_for_comparison
   80 ns  len(empty_json) == empty_json_length

Or perhaps you could zip the lists and check them in parallel, so you don't have to check the empty lists entirely.

Benchmark code (Try it online!):

from timeit import repeat

setup = '''
import numpy as np
full = [0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.05, 0.05, 0.05, 0.05, 0.075, 0.075, 0.075, 0.075, 0.1, 0.125, 0.125, 0.125, 0.15, 0.15, 0.15, 0.15, 0.175, 0.2, 0.175, 0.175, 0.175, 0.175, 0.175, 0.175, 0.15, 0.125, 0.125, 0.1, 0.075, 0.075, 0.05, 0.025, 0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, -0.025, 0.0, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, 0.0, 0.025, 0.1, 0.15, 0.175, 0.075, 0.05, 0.15, 0.35, -0.375, -0.325, -0.275, -0.2, -0.15, -0.1, -0.05, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
empty = [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
full_json = '[0.025, 0.025, 0.025, 0.025, 0.025, 0.025, 0.05, 0.05, 0.05, 0.05, 0.075, 0.075, 0.075, 0.075, 0.1, 0.125, 0.125, 0.125, 0.15, 0.15, 0.15, 0.15, 0.175, 0.2, 0.175, 0.175, 0.175, 0.175, 0.175, 0.175, 0.15, 0.125, 0.125, 0.1, 0.075, 0.075, 0.05, 0.025, 0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, 0.0, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, -0.025, 0.0, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, 0.0, -0.025, -0.025, 0.0, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, -0.025, -0.025, -0.025, -0.025, -0.05, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, -0.025, 0.0, -0.025, 0.0, 0.025, 0.1, 0.15, 0.175, 0.075, 0.05, 0.15, 0.35, -0.375, -0.325, -0.275, -0.2, -0.15, -0.1, -0.05, -0.025, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]'
empty_json = '[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]'
empty_json_for_comparison = empty_json[::-1][::-1]
empty_json_length = len(empty_json)
'''

codes = [
    'np.any({xs})',
    'max({xs})',
    'any(x != 0 for x in {xs})',
    'any({xs})',
    "'5' in {xs}_json",
    "{xs}_json == empty_json_for_comparison",
    "len({xs}_json) == empty_json_length",
]

for xs in 'full', 'empty':
    for _ in range(3):
        for code in codes:
            code = code.format(xs=xs)
            number = 1000
            t = min(repeat(code, setup, number=number)) / number
            print('] ns ' % (t * 1e9), code)
        print()

CodePudding user response：

It may be better to explicitly compare with 0 rather than treating the elements as booleans.

any(x != 0 for x in json[self.channel_number])