Skip to main content
better store version at the beginning of the file for version sniffing
Source Link

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes? (loading of older files with structural differences)

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

//version=1 public class MySettingsFile1 { public static final int VERSION = 1;  public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes? (loading of older files with structural differences)

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1;  public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes? (loading of older files with structural differences)

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

//version=1 public class MySettingsFile1 { public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

added 241 characters in body; deleted 14 characters in body
Source Link

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes? (loading of older files with structural differences)

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes?

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes? (loading of older files with structural differences)

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

added 241 characters in body; deleted 14 characters in body
Source Link

For a while now I have been toying with the idea of using source code as a file storage format.

My biggest question mark ist how to support format version changes.My question: How to support format version changes?

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

For a while now I have been toying with the idea of using source code as a file storage format.

My biggest question mark ist how to support format version changes.

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

For a while now I have been toying with the idea of using source code as a file storage format.

My question: How to support format version changes?

An example would be a settings file.

JSON:

{ "version": 1, "email": "[email protected]" } 

Then there is typically a manual or automatic parsing/deserialization/binding step that maps the textual information to a typed model in memory, e.g. an object of type Settings:

public class Settings { public String email; } 

Using source code as storage format, the settings file would look like this:

public class MySettingsFile1 { public static final int VERSION = 1; public static Settings load() { Settings s = new Settings(); s.email = "[email protected]"; return s; } } 

The settings file could then be loaded by leveraging the class loading mechanism:

String className = "MySettingsFile1"; String source = ... (read settings file to string) Class<?> clazz = new CustomClassLoader().compileAndLoad(className, source); Settings settings = (Settings) clazz.getMethod("load").invoke(null); System.out.println(settings.email); 

I already know out how to load a class from a String.

Pros

  • no parsing / binding code required
  • text format (human readable)
  • concise (verbosity comparable to JSON)
  • supports comments

Cons

  • how to support different versions and upgrade from older versions?
  • security (sandbox the loading code?)

Edit: The real use case I have in mind are files that are only modified by a custom GUI editor, and then compiled and packaged with the application (no security concerns here). These files are already under version control (XML currently).

Source Link
Loading