11
$\begingroup$

For efficiency reasons I prefer to use DumpSave instead of Save.

For ease of access I prefer to save symbols in DumpSaved files inside Global` context.

But when my code evolved and I moved it inside packages I found a lot of problems to read symbols from those DumpSaved files so that read & write process

  • allows for use saved variables inside my package (and its context)
  • does not affect variables in Global`
  • When user loads the file directly with Get, bypassing my package (and perhaps not even loading it), the symbol is available in Global` context for him or her
  • symbol name is not hard-coded into function (but of course it must hard-coded into the file itself :-( )

Simply put: I want to use DumpSave & Get the way I use Export & Import, but with efficiency and flexibility benefits.

I come up with the following code, but it still messes the Global context and has the symbol name hardcoded (Global`myglobalname):

SaveMySymbol[object_,path_String]:= Block[{}, OwnValues[Global`myglobalname] = HoldPattern[Global`myglobalname] :> object; DumpSave[ path<>".mx", Global`myglobalname]]; LoadMySymbol[path_String]:= Block[{strfullpath}, strfullpath = path<>".mx"; If[FileExistsQ[strfullpath], Get[strfullpath]; Global`myglobalname, Null]]

I guess the problem with messing the Global` context can be avoided by caching the maybe existing definition of Global`myglobalname symbol and returning it back after Get. But the code look already awfully complex (it took me a full day to figure out the trick with OwnValues) and I suspect that there must be an easy way... Well, so far there always was one with Mathematica...

Improve this question
$\endgroup$
12
  • $\begingroup$ Welcome to Mathematica.SE. Your question alludes to and then disallows some possible solutions. Would you consider using a context other than Global` for example? $\endgroup$ Commented Mar 22, 2012 at 22:53
  • 1
    $\begingroup$ All ruled out. I specifically need DumpSave. It is a huge amount of data, and it takes over 200MB and a few minutes to write it in this optimizied format. $\endgroup$ Commented Mar 22, 2012 at 23:15
  • 1
    $\begingroup$ I don't use DumpSave any more because it saves the symbol names together with their definitions, and it's a pain to manage that. Symbols names from the MX might conflict with existing ones, and if you forget what the symbols names were, it's not that convenient to retrieve them. Why don't you just use Export and Import? It works fine with MX files (the DumpSave format), but it doesn't save symbol names. Export["data.mx", data, "MX"]. $\endgroup$ Commented Mar 23, 2012 at 7:29
  • 1
    $\begingroup$ @Szabolcs It is a different paradigm. We already discussed this in comments to this answer, where I tried to describe the circumstances where using DumpSave with named symbols is actually advantageous. Basically, if you choose your own (particularly OO-style, and stateful ones, e.g. classes / structs) data structures over built-in ones, using DumpSave has advantages. $\endgroup$ Commented Mar 23, 2012 at 8:08
  • 1
    $\begingroup$ I was just unaware of this simple fact... I assumed, that if it was possible, it would be mentioned in documentation of DumpSave. So I didn't really check it. And well, I've been using Mathematica for over 10 years... $\endgroup$ Commented Mar 23, 2012 at 8:49

1 Answer 1

Reset to default
10
$\begingroup$

To show how this is possible:

Simply put: I want to use DumpSave & Get the way I use Export & Import, but with efficiency and flexibility benefits.

You can simply use Import and Export. They do support the same format that DumpSave uses, and they give you the same performance. But they don't save symbol names.

Export["data.mx", data, "MX"] data = Import["data.mx"]

MX files are not portable between architectures though, so you may consider using Compressed strings, as described here. This is several times slower, but it's still quite fast compared to any other alternative and it's portable.

If you ever need to save InterpolatingFunctions, please be aware of this problem.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.