Skip to main content
Mathematica

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

10
  • 2
    $\begingroup$ How much RAM do you have? $\endgroup$ Commented Oct 11, 2018 at 12:14
  • 1
    $\begingroup$ Hm. Files of this size have to be processes in batches. And indeed ReadList, ReadLine etc. should be the tools of choice. See here for a solution based on ReadList. It can certainly be adapted to load only chunks of the whole data set. See also the three-argument version of ReadList. $\endgroup$ Commented Oct 11, 2018 at 12:14
  • $\begingroup$ I have 8GB RAM... $\endgroup$ Commented Oct 11, 2018 at 12:18
  • 1
    $\begingroup$ can you give a minimal working example of the kind of data in each filed (only a few lines) and the intended operations? What is the code you have tried so far? $\endgroup$ Commented Oct 11, 2018 at 13:46
  • 2
    $\begingroup$ Recall that there is noticable overhead for importing data into Mathematica as well. A file that is x bytes might take 3x bytes of memory to import (the specifics here vary by file format). $\endgroup$ Commented Oct 11, 2018 at 22:55