Why is list comprehension slower than a for loop?-CodePudding

Why is the for loop appending faster than list comprehension For Loop Time: 7.214778099999876 List Comprehension Time: 7.4003780000002735

Code 1:

import timeit

mycode = '''
new_list=[]
x=[1,2,3,4,5]
for obj in x:
    if obj %2==0:
        new_list.append(obj)
'''
 
print (timeit.timeit(stmt = mycode,
                     number = 10000000))

Code 2:

import timeit

mycode = '''
x=[1,2,3,4,5]
new_list=[obj for obj in x if obj %2==0]
'''

print (timeit.timeit(stmt = mycode,
                     number = 10000000))

I expected the for loop to be slower than list comprehension when it came to appending repeatedly but that was not the case.

CodePudding user response：

This is because the list comprehension version has a bigger startup overhead. The bigger the list the faster the list comprehension is compared to the basic loop. Indeed, with x=list(range(15)) the list comprehension is 10% faster on my machine as opposed to 10% slower for the provided input. With x=list(range(1000)), the list comprehension is 25% faster than the other version.

CodePudding user response：

If you disassemble these python functions, you will see, that list comprehension compiles to a separate code object with it's own costs.

And if you trace for memory allocations, they are the same.

You can check allocations with the following code

import tracemalloc
tracemalloc.start()

def a():
    new_list = []
    x = [1, 2, 3, 4, 5]
    for obj in x:
        if obj % 2 == 0:
            new_list.append(obj)


def b():
    x = [1, 2, 3, 4, 5]
    new_list = [obj for obj in x if obj % 2 == 0]

a()
b()

snapshot = tracemalloc.take_snapshot()
top_stats = snapshot.statistics('lineno')

print("[ Top 10 ]")
for stat in top_stats[:10]:
    print(stat)

Results in

[ Top 10 ]
xxx:20: size=144 B, count=1, average=144 B
xxx:12: size=144 B, count=1, average=144 B

And bytecode with

import dis
print(dis.dis(a))
print('----------------')
print(dis.dis(b))

You will see something like

Disassembly of <code object <listcomp> at 0x1012d1790, file "xxx", line 18>:
 18           0 BUILD_LIST               0
              2 LOAD_FAST                0 (.0)
        >>    4 FOR_ITER                10 (to 26)
              6 STORE_FAST               1 (obj)
              8 LOAD_FAST                1 (obj)
             10 LOAD_CONST               0 (2)
             12 BINARY_MODULO
             14 LOAD_CONST               1 (0)
             16 COMPARE_OP               2 (==)
             18 POP_JUMP_IF_FALSE        2 (to 4)
             20 LOAD_FAST                1 (obj)
             22 LIST_APPEND              2
             24 JUMP_ABSOLUTE            2 (to 4)
        >>   26 RETURN_VALUE

In case of b, but not in case of a, which has a lot of extra commands.