How to convert subprocesss.check_output() to list from string?

Question

When I run this subprocess cmd, the type returned is a string, even though there are 5 lines printed to console

s3_ls = subprocess.check_output(["aws", "s3", "ls", s3_loc, "-- recursive", "--profile", "RoleName"]) 2016-11-17 23:34:45 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/category 2016-11-17 23:34:46 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/ 2016-12-05 17:37:25 234059 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/entity.01.csv.gz 2016-11-17 23:34:47 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/ 2016-12-05 17:37:31 109015 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/entity.02.csv.gz

How can I process this output so it's a list of 5 strings instead of just a big long one? I tried wrapping it in list(my subprocess cmd), but that just splits everything in to a single character element.

What I want to do, is have a list of files returned after running the python subprocess cmd above

quapka · Accepted Answer · 2016-12-19 21:15:03Z

3

What about splitting it?

s3_ls.split('\n')

answered Dec 19, 2016 at 21:15

quapka

2,9495 gold badges23 silver badges40 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Thom Rogers Over a year ago

I didn't think it had newlines in it because it was type str and showed a length that wa sobviously counting chars, not lines. So I never thought to try this. Thanks.

quapka Over a year ago

I don't know how it was obvious. Try len(s3_ls) and compare it to len(s3_ls.replace('\n','')).

David542 · Accepted Answer · 2016-12-19 21:16:14Z

I'm not sure how you can change the output given by the s3 command, but given that the output is one string. You could split it up into the filenames or entire line by doing:

>>> re.findall(r'dtop.+', s) ['dtop_dir/fldr_01/fldr_02/holding/date4y2m/category', 'dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/', 'dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/entity.01.csv.gz', 'dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/', 'dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/entity.02.csv.gz']

Or:

>>> re.split(r'\n', s) # or s.split('\n') would be the same here ['2016-11-17 23:34:45 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/category', '2016-11-17 23:34:46 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/', '2016-12-05 17:37:25 234059 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.01/entity.01.csv.gz', '2016-11-17 23:34:47 0 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/', '2016-12-05 17:37:31 109015 dtop_dir/fldr_01/fldr_02/holding/date4y2m/entity.02/entity.02.csv.gz']

Collectives™ on Stack Overflow

How to convert subprocesss.check_output() to list from string?

2 Answers 2

2 Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

Comments

Related