python subprocess newlines are escaped

Question

I'm running an mysql query in the command line using subprocess.Popen

process = subprocess.Popen(conarray, shell=False, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) r = process.stdout.readlines() stdout = [x.decode('utf8').rstrip() for x in r]

later I write the output in a file

f = open(file_name, 'w+', encoding='utf8') f.write(templates[idx])

It works fine but for some reason all newlines(\n) and tabs(\t) are escaped.

\t<div id="container">\n\t\t<a name="top"

Any idea on how I can fix that ?

the value is already escaped at that point, so i'm guessing it has to be something from the query [mysql, '--default-character-set=utf8', '-u', user, '-h', host, dbname, "-e %s" % query] — Mihai Vilcu
– Mihai Vilcu, Commented Aug 9, 2014 at 11:50
it's basically stdout the full code is here github.com/ionutvmi/SublimeMybbTplEditor/blob/master/… — Mihai Vilcu
– Mihai Vilcu, Commented Aug 9, 2014 at 11:56
with .decode('unicode_escape') it seems to parse the new lines correctly but it doesn't convert chars like ńńńńń — Mihai Vilcu
– Mihai Vilcu, Commented Aug 9, 2014 at 12:01

falsetru · Accepted Answer · 2014-08-09 12:15:07Z

If you print the whole sequence object, repr(list_object) is printed. That's the way Python represent it.

>>> lst = ['\t<div id="container">\n\t\t<a name="top"'] >>> print(lst) ['\t<div id="container">\n\t\t<a name="top"'] >>> print(lst[0]) <div id="container"> <a name="top" >>>

Community · Accepted Answer · 2017-05-23 12:31:22Z

1

It turns out that in order to make them work I also needed to apply unicode-escape on that string.
But unicode-escape does NOT work in general.

>>> s = 'naïve \\t test' >>> print(s.encode('utf-8').decode('unicode_escape')) naÃ¯ve test

The best solution I could find is described in this answer

edited May 23, 2017 at 12:31

CommunityBot

11 silver badge

answered Aug 10, 2014 at 6:16

Mihai Vilcu

1,96718 silver badges26 bronze badges

2 Comments

user2357112 Over a year ago

Your string appears to have actual backslashes in it, rather than special characters merely represented with backslashes in the escape sequences. It probably went through an extra layer of escaping at some point. You'll need to figure out why that's happening and fix it.

Mihai Vilcu Over a year ago

here is the code to try it out on your computer gist.github.com/ionutvmi/a63e84c0402db7a266fd

Collectives™ on Stack Overflow

python subprocess newlines are escaped

2 Answers 2

Comments

2 Comments

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

2 Comments

Linked

Related