How to fix the python logging order when there are multiple generators calling a common logger-CodePudding

There are a list of generators gen_list = [gen_1, gen_2, gen_3].

Each gen_x has a logging statement that put a log line in a common logfile.

For example,

gen_1 puts a log line "log 1"

gen_2 puts a log line "log 2"

gen_3 puts a log line "log 3"

After gen_list is executed, the log_file looks like the following.

log 1

log 2

log 3

log 2

log 1

log 2

log 3

...

The execution order of generators in the list is alternating. (1, 2, 3, 3, 2, 1, 1, 2, 3, ...)

Is there any way to fix the order always forward-manner? (1, 2, 3, 1, 2, 3, 1, 2, 3, ...)

For example, I want the log_file to be

log 1

log 2

log 3

log 1

log 2

log 3

...

CodePudding user response：

You can use zip() to simulate cycling through the iterators, and then use chain.from_iterable() from itertools to chain the tuples from zip() together. This allows you to read out one element from one generator at a time. One notable advantage of this method is that you can use arbitrarily many generators.

Here is an example:

from itertools import chain

def gen_1():
    for i in range(10):
        yield "log1"

def gen_2():
    for i in range(10):
        yield "log2"

def gen_3():
    for i in range(10):
        yield "log3"

gen_list = [gen_1, gen_2, gen_3]

for elem in chain.from_iterable(zip(*[gen() for gen in gen_list])):
    print(elem)

After using zip(), you'll produce something like:

[("log1", "log2", "log3"), ("log1", "log2", "log3"), ...]

Then, using chain.from_iterable() flattens these tuples to give us a sequence like:

"log1", "log2", "log3", "log1", "log2", "log3", ...

CodePudding user response：

I'm not sure you're providing all the information to explain your problem. This script:

import logging

def gen_1():
    for i in range(10):
        logging.debug('log 1: %d' % i)
        yield "log1"

def gen_2():
    for i in range(10):
        logging.debug('log 2: %d' % i)
        yield "log2"

def gen_3():
    for i in range(10):
        logging.debug('log 3: %d' % i)
        yield "log3"

gen_list = [gen_1(), gen_2(), gen_3()]

logging.basicConfig(level=logging.DEBUG, format='%(message)s')

def main():
    for i in range(10):
        v = [next(gen_list[i]) for i in range(3)]

if __name__ == '__main__':
    main()

prints, when run (with Python 3),

log 1: 0
log 2: 0
log 3: 0
log 1: 1
log 2: 1
log 3: 1
log 1: 2
log 2: 2
log 3: 2
log 1: 3
log 2: 3
log 3: 3
log 1: 4
log 2: 4
log 3: 4
log 1: 5
log 2: 5
log 3: 5
log 1: 6
log 2: 6
log 3: 6
log 1: 7
log 2: 7
log 3: 7
log 1: 8
log 2: 8
log 3: 8
log 1: 9
log 2: 9
log 3: 9