13

Does converting from the mutable bytearray type to the non-mutable bytes type incur a copy? Is there any cost associated with it, or does the interpreter just treat it as an immutable byte sequence, like casting a char* to a const char* const in C++?

ba = bytearray() ba.extend("some big long string".encode('utf-8')) # Is this conversion free or expensive? write_bytes(bytes(ba))

Does this differ between Python 3 where bytes is its own type and Python 2.7 where bytes is just an alias for str?

Improve this question
4
  • All operations involve some cost. You could look at the source, or you can use timing tests to see if the time increases linearly with the problem size (as would happen if a copy was made). Commented Mar 8, 2016 at 23:25
  • 1
    I'm fairly sure that converting bytearray to bytes incurs a copy. This is because if the new bytes points to the same backing array as the bytearray, then it wouldn't be truly immutable. Commented Mar 8, 2016 at 23:28
  • 2
    Note that if you want a view on the contents of a bytearray without making a copy, you can use a memoryview for the purpose. The caveat is that changes to the bytearray data will change the data in the memoryview, and that the bytearray cannot be resized (no appends, pops, resizing slice assignments, etc.) for as long as any exported buffers (of which memoryview is the most common type created in Python level code) exist. Commented Mar 9, 2016 at 1:45
  • In Python 2 you can also use buffer() to make a read-only buffer out of a bytearray without actually performing a copy Commented Dec 16, 2021 at 16:11

2 Answers 2

Reset to default
21

A new copy is created, the buffer is not shared between the bytesarray and the new bytes object, in either Python 2 or 3.

You couldn't share it, as the bytesarray object could still be referenced elsewhere and mutate the value.

For the details, see the bytesobject.c source code, where the buffer protocol is used to create a straight up copy of the data (via PyBuffer_ToContiguous()).

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

9

Martjin is right. I just wanted to back that answer up with the cpython source.

Looking at the source for bytes here, first bytes_new is called, which will call PyBytes_FromObject, which will call _PyBytes_FromBuffer, which creates a new bytes object and calls PyBuffer_ToContiguous (defined here). This calls buffer_to_contiguous, which is a memory copy function. The comment for the function reads:

Copy src to a contiguous representation. order is one of 'C', 'F' (Fortran) or 'A' (Any). Assumptions: src has PyBUF_FULL information, src->ndim >= 1, len(mem) == src->len.

Thus, a call to bytes with a bytearray argument will copy the data.

Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.