4

In your experience, what's the best way to store a 2-dimensional array into a text file? My matrix is pretty huge, but let's use a small 3 x 3 matrix as an example. One could store it, quite literally, like this (to the .txt file):

3 7 5 2 9 8 8 0 1

with a space between elements of the same row, and a newline character to separate the rows themselves from each other. And then if you want to use this saved matrix with a different python file, you'd just parse accordingly.

However, is there a much better (for instance, a way that requires few lines of code) way to store and retrieve a large matrix?

[EDIT]: Right, now I have a list of lists (or, an array of arrays), but I'm open to the idea of using numpy if that makes things a lot easier

Improve this question
4
  • 2
    You say "2-dimensional array" and "matrix"; are you using numpy, or do you really have a list of lists? Commented Mar 23, 2014 at 3:36
  • Indent the numbers with four spaces to format as code. This is as easy as selecting the relevant text and clicking the {} button in the editor. Commented Mar 23, 2014 at 3:39
  • @DSM I edited the question, to answer your question Commented Mar 23, 2014 at 3:41
  • Note: list and array aren't synonyms in Python. list is a built-in type; there's an array.array type, also built in; and there's numpy.ndarray, also often called array. Commented Mar 23, 2014 at 3:43

4 Answers 4

Reset to default
1

If it's a simple array of arrays, one option is to store it as json with json.dump(open('file.dat', 'w')) and load it with json.load(open('file.dat')). This has its pros and cons. (Readable, but not the most efficient.)

If using numpy is an option, you should use it. It's much more efficient at holding, representing and serializing large arrays.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

1

You can do this using the csv module:

import csv l = [[3,7,5], [2,9,8], [8,0,1]] with open('matrix.txt', 'wb') as csvfile: matrixwriter = csv.writer(csvfile, delimiter=' ') for row in l: matrixwriter.writerow(row) l2 = [] with open('matrix.txt', 'rb') as csvfile: matrixreader = csv.reader(csvfile, delimiter=' ') for row in matrixreader: l2.append(row) print(l2)
Improve this answer

2 Comments

would you say this is better than using the numpy method?
The csv module is part of the Python Standard Library and numpy isn't. Whether that matters to you depends on just where you'll be using it.
1

If you are using numpy, there are several routines listed under the io category that should help. An example of one of them follows:

>>> """ Assuming your matrix is called x """ >>> from tempfile import TemporaryFile >>> outfile = TemporaryFile() >>> np.save(outfile, x)

To reload this, you need the load function:

>>> x2 = np.load(outfile)

And your array should be copied into x2.

An example of creating a numpy array from a 2-dimensional python array is the following:

x = np.array([2, 3, 1, 0])

If you need further help, do leave a comment.

Improve this answer

2 Comments

So I'm guessing you would initialize 'x' as a numpy matrix, or would you do it the naive way by just creating an array of arrays?
Please see the edit of my answer for assistance in creating the correct structure...
0

Personally I would use joblib for storing binary representation of arrays. It also allows for the use of a compress argument to cut down on size.

from joblib import dump dump(arr, filename, compress=3)
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.