Categories
iteration python

How to remove items from a list while iterating?

933

I’m iterating over a list of tuples in Python, and am attempting to remove them if they meet certain criteria.

for tup in somelist:
    if determine(tup):
         code_to_remove_tup

What should I use in place of code_to_remove_tup? I can’t figure out how to remove the item in this fashion.

1

  • Most answers on this page don’t really explain why removing elements while iterating over a list produces strange results, but the accepted answer in this question does, and is probably a better dupe for beginners who encounter this issue for the first time.

    – ggorlen

    Dec 6, 2019 at 23:34

1032

You can use a list comprehension to create a new list containing only the elements you don’t want to remove:

somelist = [x for x in somelist if not determine(x)]

Or, by assigning to the slice somelist[:], you can mutate the existing list to contain only the items you want:

somelist[:] = [x for x in somelist if not determine(x)]

This approach could be useful if there are other references to somelist that need to reflect the changes.

Instead of a comprehension, you could also use itertools. In Python 2:

from itertools import ifilterfalse
somelist[:] = ifilterfalse(determine, somelist)

Or in Python 3:

from itertools import filterfalse
somelist[:] = filterfalse(determine, somelist)

10

  • 6

    Can you make it faster if you know only a few will be deleted, i.e., only delete those and leave the others in-place rather than re-writing them?

    Apr 20, 2011 at 19:25

  • 47

    What if my list is huge and can’t afford making a copy?

    – jpcgt

    Nov 15, 2014 at 23:43

  • 24

    @jpcgt You should use somelist[:] = (x for x in somelist if determine(x)) this will create generator that may not create any unnecessary copies.

    Apr 29, 2015 at 14:54


  • 9

    @RostislavKondratenko: list_ass_slice() function that implements somelist[:]= calls PySequence_Fast() internally. This function always returns a list i.e., @Alex Martelli’s solution that already uses a list instead of a generator is most probably more efficient

    – jfs

    May 7, 2015 at 20:48

  • 8

    Would you care to explain what the differences are between assigning the list comprehension to the list and list clone please? Wouldn’t the original list somelist be mutated in both methods?

    – Bowen Liu

    Sep 24, 2018 at 19:06

672

The answers suggesting list comprehensions are ALMOST correct — except that they build a completely new list and then give it the same name the old list as, they do NOT modify the old list in place. That’s different from what you’d be doing by selective removal, as in @Lennart’s suggestion — it’s faster, but if your list is accessed via multiple references the fact that you’re just reseating one of the references and NOT altering the list object itself can lead to subtle, disastrous bugs.

Fortunately, it’s extremely easy to get both the speed of list comprehensions AND the required semantics of in-place alteration — just code:

somelist[:] = [tup for tup in somelist if determine(tup)]

Note the subtle difference with other answers: this one is NOT assigning to a barename – it’s assigning to a list slice that just happens to be the entire list, thereby replacing the list contents within the same Python list object, rather than just reseating one reference (from previous list object to new list object) like the other answers.

8

  • 1

    How do I do the same sliced assignment with a dict? In Python 2.6?

    – PaulMcG

    Mar 25, 2011 at 19:29

  • 11

    @Paul: Since dicts are unordered, slices are meaningless for dicts. If your want to replace the contents of dict a by the contents of dict b, use a.clear(); a.update(b).

    Apr 1, 2011 at 23:51

  • 1

    Why can ‘reseating’ one of the references by replacing what the variable refers to cause bugs? It seems like that would only be a potential problem in multi-threaded applications, not single-threaded.

    Aug 7, 2011 at 22:59

  • 72

    @Derek x = ['foo','bar','baz']; y = x; x = [item for item in x if determine(item)]; This reassigns x to the result of the list comprehension, but y still refers to the original list ['foo','bar','baz']. If you expected x and y to refer to the same list, you may have introduced bugs. You prevent this by assigning to a slice of the entire list, as Alex shows, and I show here: x = ["foo","bar","baz"]; y = x; x[:] = [item for item in x if determine(item)];. The list is modified in place. ensuring that all references to the list (both x and y here) refer to the new list.

    Nov 15, 2011 at 19:38


  • in fact, using filter function too creates a new list, does not modify elements in place… only olist[:] = [i for i in olist if not dislike(i)]

    Jul 13, 2016 at 8:48

375

You need to take a copy of the list and iterate over it first, or the iteration will fail with what may be unexpected results.

For example (depends on what type of list):

for tup in somelist[:]:
    etc....

An example:

>>> somelist = range(10)
>>> for x in somelist:
...     somelist.remove(x)
>>> somelist
[1, 3, 5, 7, 9]

>>> somelist = range(10)
>>> for x in somelist[:]:
...     somelist.remove(x)
>>> somelist
[]

13

  • 18

    @Zen Because the second one iterates over a copy of the list. So when you modify the original list, you do not modify the copy that you iterate over.

    Jun 18, 2014 at 13:47


  • 3

    What’s better in doing somelist[:] compared to list(somelist) ?

    Feb 4, 2015 at 10:01

  • 3

    list(somelist) will convert an iterable into a list. somelist[:] makes a copy of an object that supports slicing. So they don’t necessarily do the same thing. In this case I want to make a copy of the somelistobject, so I use [:]

    Feb 5, 2015 at 12:21


  • 46

    Note to anyone reading this, this is VERY slow for lists. remove() has to go over the WHOLE list for every iteration, so it will take forever.

    – vitiral

    Feb 11, 2015 at 23:22

  • 14

    Big O time doesn’t matter when dealing with lists of only a dozen items. Often clear and simple for future programmers to understand is far more valuable than performance.

    – Steve

    Aug 6, 2016 at 17:21