Categories
clone copy list python reference

How do I clone a list so that it doesn’t change unexpectedly after assignment?

3097

While using new_list = my_list, any modifications to new_list changes my_list every time. Why is this, and how can I clone or copy the list to prevent it?

4

3863

new_list = my_list doesn’t actually create a second list. The assignment just copies the reference to the list, not the actual list, so both new_list and my_list refer to the same list after the assignment.

To actually copy the list, you have several options:

  • You can use the builtin list.copy() method (available since Python 3.3):
new_list = old_list.copy()
  • You can slice it:
new_list = old_list[:]

Alex Martelli‘s opinion (at least back in 2007) about this is, that it is a weird syntax and it does not make sense to use it ever. 😉 (In his opinion, the next one is more readable).

  • You can use the built in list() function:
new_list = list(old_list)
import copy
new_list = copy.copy(old_list)

This is a little slower than list() because it has to find out the datatype of old_list first.

  • If you need to copy the elements of the list as well, use generic copy.deepcopy():
import copy
new_list = copy.deepcopy(old_list)

Obviously the slowest and most memory-needing method, but sometimes unavoidable. This operates recursively; it will handle any number of levels of nested lists (or other containers).

Example:

import copy

class Foo(object):
    def __init__(self, val):
         self.val = val

    def __repr__(self):
        return f'Foo({self.val!r})'

foo = Foo(1)

a = ['foo', foo]
b = a.copy()
c = a[:]
d = list(a)
e = copy.copy(a)
f = copy.deepcopy(a)

# edit orignal list and instance 
a.append('baz')
foo.val = 5

print(f'original: {a}\nlist.copy(): {b}\nslice: {c}\nlist(): {d}\ncopy: {e}\ndeepcopy: {f}')

Result:

original: ['foo', Foo(5), 'baz']
list.copy(): ['foo', Foo(5)]
slice: ['foo', Foo(5)]
list(): ['foo', Foo(5)]
copy: ['foo', Foo(5)]
deepcopy: ['foo', Foo(1)]

6

  • 4

    As @Georgy points out correctly in the answer below, any changes to the new_list values will also change the values in my_list. So actually the copy.deepcopy() method is the only real copy without reference to the original list and it’s values.

    – moojen

    Dec 21, 2020 at 17:27

  • You’re right, it was edited by you, but posted by @cryo Sorry for the mixup!

    – moojen

    Dec 21, 2020 at 17:52

  • Which one is fastest?

    – uuu777

    May 4, 2021 at 18:10

  • I was having the same issue with a list of json (each element of a list was a json) and the only one that worked was new_list = copy.deepcopy(old_list) ; I’m writing this since anyone can encounter the same issue. Thanks!

    – Tom

    May 11, 2021 at 15:23

  • 2

    +1 for slicing [:] it is a simple and compact syntax and it does make sense to use it every time you need to copy a list and can avoid a deepcopy

    – Raf

    Sep 28, 2021 at 11:02

725

Felix already provided an excellent answer, but I thought I’d do a speed comparison of the various methods:

  1. 10.59 sec (105.9 µs/itn) – copy.deepcopy(old_list)
  2. 10.16 sec (101.6 µs/itn) – pure Python Copy() method copying classes with deepcopy
  3. 1.488 sec (14.88 µs/itn) – pure Python Copy() method not copying classes (only dicts/lists/tuples)
  4. 0.325 sec (3.25 µs/itn) – for item in old_list: new_list.append(item)
  5. 0.217 sec (2.17 µs/itn) – [i for i in old_list] (a list comprehension)
  6. 0.186 sec (1.86 µs/itn) – copy.copy(old_list)
  7. 0.075 sec (0.75 µs/itn) – list(old_list)
  8. 0.053 sec (0.53 µs/itn) – new_list = []; new_list.extend(old_list)
  9. 0.039 sec (0.39 µs/itn) – old_list[:] (list slicing)

So the fastest is list slicing. But be aware that copy.copy(), list[:] and list(list), unlike copy.deepcopy() and the python version don’t copy any lists, dictionaries and class instances in the list, so if the originals change, they will change in the copied list too and vice versa.

(Here’s the script if anyone’s interested or wants to raise any issues:)

from copy import deepcopy

class old_class:
    def __init__(self):
        self.blah="blah"

class new_class(object):
    def __init__(self):
        self.blah="blah"

dignore = {str: None, unicode: None, int: None, type(None): None}

def Copy(obj, use_deepcopy=True):
    t = type(obj)

    if t in (list, tuple):
        if t == tuple:
            # Convert to a list if a tuple to
            # allow assigning to when copying
            is_tuple = True
            obj = list(obj)
        else:
            # Otherwise just do a quick slice copy
            obj = obj[:]
            is_tuple = False

        # Copy each item recursively
        for x in xrange(len(obj)):
            if type(obj[x]) in dignore:
                continue
            obj[x] = Copy(obj[x], use_deepcopy)

        if is_tuple:
            # Convert back into a tuple again
            obj = tuple(obj)

    elif t == dict:
        # Use the fast shallow dict copy() method and copy any
        # values which aren't immutable (like lists, dicts etc)
        obj = obj.copy()
        for k in obj:
            if type(obj[k]) in dignore:
                continue
            obj[k] = Copy(obj[k], use_deepcopy)

    elif t in dignore:
        # Numeric or string/unicode?
        # It's immutable, so ignore it!
        pass

    elif use_deepcopy:
        obj = deepcopy(obj)
    return obj

if __name__ == '__main__':
    import copy
    from time import time

    num_times = 100000
    L = [None, 'blah', 1, 543.4532,
         ['foo'], ('bar',), {'blah': 'blah'},
         old_class(), new_class()]

    t = time()
    for i in xrange(num_times):
        Copy(L)
    print 'Custom Copy:', time()-t

    t = time()
    for i in xrange(num_times):
        Copy(L, use_deepcopy=False)
    print 'Custom Copy Only Copying Lists/Tuples/Dicts (no classes):', time()-t

    t = time()
    for i in xrange(num_times):
        copy.copy(L)
    print 'copy.copy:', time()-t

    t = time()
    for i in xrange(num_times):
        copy.deepcopy(L)
    print 'copy.deepcopy:', time()-t

    t = time()
    for i in xrange(num_times):
        L[:]
    print 'list slicing [:]:', time()-t

    t = time()
    for i in xrange(num_times):
        list(L)
    print 'list(L):', time()-t

    t = time()
    for i in xrange(num_times):
        [i for i in L]
    print 'list expression(L):', time()-t

    t = time()
    for i in xrange(num_times):
        a = []
        a.extend(L)
    print 'list extend:', time()-t

    t = time()
    for i in xrange(num_times):
        a = []
        for y in L:
            a.append(y)
    print 'list append:', time()-t

    t = time()
    for i in xrange(num_times):
        a = []
        a.extend(i for i in L)
    print 'generator expression extend:', time()-t

3

  • Does it mean that append and list comprehension are the best options?

    – uuu777

    May 4, 2021 at 18:24

  • I have a cache containing a list of classes, I want to take lock, copy out the list, release lock. I hope that it is enough to use built-in copy to protect copied out list from changing when cached copy is changed.

    – uuu777

    May 4, 2021 at 18:32

  • I keep on coming back to this answer to make sure that I am using the most efficient method. What is the easiest way to test this? Or is there a database with all of the best ways to minimise run time?

    Dec 9, 2021 at 0:34

178

I’ve been told that Python 3.3+ adds the list.copy() method, which should be as fast as slicing:

newlist = old_list.copy()

4