Categories
list loops python

Strange result when removing item from a list while iterating over it

78

I’ve got this piece of code:

numbers = list(range(1, 50))

for i in numbers:
    if i < 20:
        numbers.remove(i)

print(numbers)

but the result I’m getting is:
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

Of course, I’m expecting the numbers below 20 to not appear in the results. Looks like I’m doing something wrong with the remove.

1

  • See also: stackoverflow.com/questions/1207406/…. I reconsidered and decided that this is not a duplicate; this question is about understanding the failure of one specific wrong way to approach the problem, while the other question is about finding correct ways.

    Jul 30 at 0:29

132

You’re modifying the list while you iterate over it. That means that the first time through the loop, i == 1, so 1 is removed from the list. Then the for loop goes to the second item in the list, which is not 2, but 3! Then that’s removed from the list, and then the for loop goes on to the third item in the list, which is now 5. And so on. Perhaps it’s easier to visualize like so, with a ^ pointing to the value of i:

[1, 2, 3, 4, 5, 6...]
 ^

That’s the state of the list initially; then 1 is removed and the loop goes to the second item in the list:

[2, 3, 4, 5, 6...]
    ^
[2, 4, 5, 6...]
       ^

And so on.

There’s no good way to alter a list’s length while iterating over it. The best you can do is something like this:

numbers = [n for n in numbers if n >= 20]

or this, for in-place alteration (the thing in parens is a generator expression, which is implicitly converted into a tuple before slice-assignment):

numbers[:] = (n for in in numbers if n >= 20)

If you want to perform an operation on n before removing it, one trick you could try is this:

for i, n in enumerate(numbers):
    if n < 20 :
        print("do something")
        numbers[i] = None
numbers = [n for n in numbers if n is not None]

3

  • Related note on for keeping an index from the Python docs docs.python.org/3.9/reference/…: “There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, e.g. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. … This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated).

    Jan 31 at 4:03


  • This is a good answer, but with the final solution, “if you want to perform an operation…”, is slightly unsatisfactory because 1) in fact there is no need to include that qualification: it is just a waste of effort to try and remove elements while iterating in a single operation, so this 2-stage solution applies in all cases, and 2) because there should be a warning that setting to None won’t always be appropriate (if some elements are meant to be None): some conventional “poison pill” value, appropriate to the case in hand, is needed.

    Mar 16 at 12:08

  • When I say that about the “waste of effort”, I am referring to the “general” solution by the way, i.e. when random elements may need removing, rather than the “specialist” case of removing elements only at the start (or only at the end), which lends itself to something simple, like your list comprehension solution…

    Mar 16 at 12:27

15

Begin at the list’s end and go backwards:

li = list(range(1, 15))
print(li)

for i in range(len(li) - 1, -1, -1):
    if li[i] < 6:
        del li[i]
        
print(li)

Result:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] 
[6, 7, 8, 9, 10, 11, 12, 13, 14]

5

  • 1

    How I wish I could +2 this answer! Elegant, easy…not entirely obfuscated.

    Dec 8, 2021 at 4:17

  • 1

    This is a very specialised answer: it is not in fact clear whether we’re meant to be looking for a general solution to the problem of how to remove elements while iterating, or how to do this exclusively when we just want to remove the first n elements of a list. The chosen answer provides the former, which is infinitely more helpful, but also the latter, in the shape of a one-line list comprehension solution.

    Mar 16 at 12:22


  • @mikerodent no it’s not. It’s pretty common when you want to modify a list while iterating over it that going backwards works

    Apr 7 at 11:49


  • 1

    @Boris you haven’t understood my comment. The OP’s question does not specify that we are removing contiguous elements (either from the start or end of the list).

    Apr 7 at 12:56

  • I still don’t understand your comment then because it doesn’t matter whether the list is shuffled or ordered, this code will still work.

    Apr 7 at 12:57


9

@senderle’s answer is the way to go!

Having said that to further illustrate even a bit more your problem, if you think about it, you will always want to remove the index 0 twenty times:

[1,2,3,4,5............50]
 ^
[2,3,4,5............50]
 ^
[3,4,5............50]
 ^

So you could actually go with something like this:

aList = list(range(50))
i = 0
while i < 20:
    aList.pop(0)
    i += 1

print(aList) #[21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

I hope it helps.


The ones below are not bad practices AFAIK.

EDIT (Some more):

lis = range(50)
lis = lis[20:]

Will do the job also.

EDIT2 (I’m bored):

functional = filter(lambda x: x> 20, range(50))

0