Python for loop efficiency

Question

I have this block of code, which works, but takes about 8 seconds to execute. I know that it is the second for loop, since there is a loop inside of a loop. However, I believe that I need both loops, because I need to cross-reference the tracks list.

Does anybody know of a way to make this function execute faster? I can't seem to see another way of writing it.

FYI : The csv file that I am using has 5570 lines, which is another reason for the function taking a "while".

Thanks in advance!

def load_library(filename) : library = open(filename, 'rb') reader = csv.reader(library, delimiter = '|') tracks = set([]) albums = set([]) albums1 = set([]) #albums1 is the set of albums which have already been added to the albums list. for row in reader : artist, track, album, genre, year = row track = Track(artist, track) track.set_album(album) tracks.add(track) library = open(filename, 'rb') reader = csv.reader(library, delimiter = '|') for row in reader : artist, track, album, genre, year = row a = Album(artist, album) for i in tracks : if str(i.album) == str(a.title) : a.add_track(i.title) if album not in albums1 : albums.add(a) albums1.add(album) return tracks, albums

After using c.Profile :

cProfile.run('load_library()') 224565 function calls in 9.776 seconds

 Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.002 0.002 9.776 9.776 <string>:1(<module>) 5570 0.001 0.000 0.001 0.000 musiclib.py:18(set_album) 11140 0.007 0.000 0.007 0.000 musiclib.py:23(__init__) 92784 0.028 0.000 0.037 0.000 musiclib.py:31(add_track) 5570 0.004 0.000 0.009 0.000 musiclib.py:6(__init__) 1 9.723 9.723 9.775 9.775 musiclib.py:71(load_library) 2 0.000 0.000 0.000 0.000 {_csv.reader} 16710 0.002 0.000 0.002 0.000 {method 'add' of 'set' objects} 92784 0.009 0.000 0.009 0.000 {method 'append' of 'list' objects} 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 2 0.000 0.000 0.000 0.000 {open}

Well, it is faster to use readline(line_num) then to use a for loop for going through each line. — user2961646
– user2961646, Commented Oct 5, 2014 at 17:44
Try profiling it with cProfile to see which parts are slow. — rlms
– rlms, Commented Oct 5, 2014 at 17:52
@AHuman Thank you, but how would I use readline(line_num) to read every line? — Evan Cooper
– Evan Cooper, Commented Oct 5, 2014 at 17:52
It is up to you. But you can just use 5570 more lines. I would just make a new python file and then just import it over and run a function with 5571 lines with the last line returning your result. That way your program looks clean. — user2961646
– user2961646, Commented Oct 5, 2014 at 17:55
@sweeneyrod I put my filename within the function because I was getting a syntax error. Nevertheless, still the same output. I edited the post with the results. — Evan Cooper
– Evan Cooper, Commented Oct 5, 2014 at 18:00

Daniel · Accepted Answer · 2014-10-05 18:07:07Z

Do this all in only one for-loop:

def load_library(filename) : library = open(filename, 'rb') reader = csv.reader(library, delimiter = '|') tracks = set([]) albums = {} for row in reader : artist, track, album, genre, year = row if album not in albums: a = Album(artist, album) albums[album] = a else: a = albums[album] a.add_track(track) track = Track(artist, track) track.set_album(album) tracks.add(track) return tracks, set(albums.values())

Collectives™ on Stack Overflow

Python for loop efficiency

1 Answer 1

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Related