Skip to main content
Mathematica

You are not logged in. Your edit will be placed in a queue until it is peer reviewed.

We welcome edits that make the post easier to understand and more valuable for readers. Because community members review edits, please try to make the post substantially better than how you found it, for example, by fixing grammar or adding additional resources and hyperlinks.

Required fields*

15
  • $\begingroup$ Wow. I am impressed. Really. It will take me some time to absorb that. But I will certainly set out the time and do that. Big +1, of course. $\endgroup$ Commented Aug 30, 2015 at 22:51
  • $\begingroup$ I have to mention, that there are types of chunks called Virtual, for which there is no caching on disk. I did not demonstrate how to use them, and also there seems to be a bug there which I still have to fix, but for example infinite Range is built with those. They can be used any time when you can generate chunks purely by calling some function of the chunk index. In the implementation of LazyTuples, there is a branch that uses virtual chunks. They might be better suited for this case. I will have to revisit the bug there though. $\endgroup$ Commented Aug 30, 2015 at 23:00
  • $\begingroup$ Also, while using DeleteFile@FileNames["*.mx", $StreamingCacheBase] will indeed delete the chunks from disk, this will leave Streaming in an invalid state. The better way is to use something like Scan[LazyListDestroy, LazyLists[]], which should (if the Streaming garbage collector works correctly, which I hope is true) then delete chunks from disk as a part of the lazy list destruction procedure. $\endgroup$ Commented Aug 30, 2015 at 23:07
  • 4
    $\begingroup$ @Leonid I suspected that best way to learn new Mathematica module is to post some code and let module author point out all mistakes ;) $\endgroup$ Commented Aug 30, 2015 at 23:25
  • 2
    $\begingroup$ I'd say you did an excellent job here, given that you had to dig a lot of things and there is no documentation. In any case, thanks a bunch for such a great non-trivial test of Streaming functionality! It is great to see this stuff being put to use. $\endgroup$ Commented Aug 30, 2015 at 23:26