27

I want to write a file. Based on the name of the file this may or may not be compressed with the gzip module. Here is my code:

import gzip filename = 'output.gz' opener = gzip.open if filename.endswith('.gz') else open with opener(filename, 'wb') as fd: print('blah blah blah'.encode(), file=fd)

I'm opening the writable file in binary mode and encoding my string to be written. However I get the following error:

File "/usr/lib/python3.5/gzip.py", line 258, in write data = memoryview(data) TypeError: memoryview: a bytes-like object is required, not 'str'

Why is my object not a bytes? I get the same error if I open the file with 'w' and skip the encoding step. I also get the same error if I remove the '.gz' from the filename.

I'm using Python3.5 on Ubuntu 16.04

Improve this question
2
  • 1
    Its the same as print(str('blah blah blah'.encode()), file=fd). python is trying to write the string representation of the bytes object. Commented Dec 2, 2016 at 0:42
  • @nic you can use the 'wt' flag and write strings directly - see my answer below Commented Jun 6, 2019 at 8:05

5 Answers 5

Reset to default
26

For me, changing the gzip flag to 'wt' did the job. I could write the original string, without "byting" it. (tested on python 3.5, 3.7 on ubuntu 16).

From python 3 gzip doc - quoting: "... The mode argument can be any of 'r', 'rb', 'a', 'ab', 'w', 'wb', 'x' or 'xb' for binary mode, or 'rt', 'at', 'wt', or 'xt' for text mode..."

import gzip filename = 'output.gz' opener = gzip.open if filename.endswith('.gz') else open with opener(filename, 'wt') as fd: print('blah blah blah', file=fd) !zcat output.gz > blah blah blah
Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

I think this answer is already a better one (even looking at votes). Shall we change this response to the "accepted" answer?
15

you can convert it to bytes like this.

import gzip with gzip.open(filename, 'wb') as fd: fd.write('blah blah blah'.encode('utf-8'))
Improve this answer

Comments

7

print is a relatively complex function. It writes str to a file but not the str that you pass, it writes the str that is the result of rendering the parameters.

If you have bytes already, you can use fd.write(bytes) directly and take care of adding a newline if you need it.

If you don't have bytes, make sure fd is opened to receive text.

Improve this answer

7 Comments

use 'fd.write(b"blah blah blah")'
@citizenSNIPS, I don't understand you comment.
add a 'b' infront of the str to signify it is a byte. I believe you want to pass a bytes-literal. [](python.org/dev/peps/pep-3112)
I do not. I only mention types, no sample data.
I see. my point was if the file was opened using wb, then the str would need to be a byte, and adding b infront would do that.
|
0

You can also use io.TextIOWrapper as intermediate buffer:

import gzip import io filename = 'output.gz' opener = gzip.open if filename.endswith('.gz') else open with opener(filename, 'wb') as fd: print('blah blah blah', file=io.TextIOWrapper(fd, write_through=True))

Output:

# For a gzip file >>> !zcat output.gz blah blah blah # For a text file >>> !cat output.txt blah blah blah
Improve this answer

Comments

-2

You can serialize it using pickle.

First serializing the object to be written using pickle, then using gzip.

To save the object:

import gzip, pickle filename = 'non-serialize_object.zip' # serialize the object serialized_obj = pickle.dumps(object) # writing zip file with gzip.open(filename, 'wb') as f: f.write(serialized_obj)

To load the object:

import gzip, pickle filename = 'non-serialize_object.zip' with gzip.open(filename, 'rb') as f: serialized_obj = f.read() # de-serialize the object object = pickle.loads(serialized_obj)
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.