C Programming/X macros and serialization

It is often necessary to send or receive complex data structures to or from another program that may run on a different architecture or may have been designed for different version of the data structures in question.

A typical example is a program that saves its state to a file on exit and then reads it back when started. The 'send' function will typically start by writing a magic identifier and version to the file or network socket and then proceed to write all the data members one by one (i.e. in serial). The 'receive' function will be nearly identical: it will read all the items one by one.

Since these two functions often follow the same pattern as the declaration of the data(structures), it would be nice if they could all be generated from a common definition.

X macros

X macros use the preprocessor to force the compiler process the same piece of text—a list of data—under different macro expansions. Each entry in the list is wrapped in a function-like macro, which has a different definition each time the list is used.

The 'X' in 'X macros' doesn't mean anything. It's taken from the most commonly used macro name for the pattern, X(...), chosen because it shortens typing when making the list.

The most common usage of X macros is to establish a list of objects based on a list of some data, then automatically generate code for each of them, using the same list.

Separate list

Usually, a separate file contains this list. As an example, let's say we want to generate several variables near the beginning of our program, then later in the program, we want to print the names and vales of those variables in the same console. We can create a stats.def like this:

X(strength) X(agility) X(intelligence) 

Then, in our C program, we make sure the X(...) macro is defined as a template for our generated code:

#define X(x) int x; #include "stats.def" #undef X // ... #define X(x) printf(#x " = %d\n", x); #include "stats.def" #undef X 

After the preprocessor runs, this is what the compiler sees:

int strength; int agility; int intelligence; // ... printf("strength" " = %d\n", strength); printf("agility" " = %d\n", agility); printf("intelligence" " = %d\n", intelligence); 

Now, whenever a new statistic is added to our program, we only have to make one edit instead of remembering to make multiple.

Inline list

If including a separate file multiple times is undesirable, the list can be defined as an object-like macro earlier in the file.

#define STATS \  X(strength) \  X(agility) \  X(intelligence) 

The #include "stats.def" directives from the previous example can then be replaced with STATS:

#define X(x) int x; STATS #undef X // ... #define X(x) printf(#x " = %d\n", x); STATS #undef X 

Higher-order X macros

One can also pass in the name of another macro that can operate on the list of values. For example:

#define STATS_FOREACH(X) \  X(strength) \  X(agility) \  X(intelligence) #define STATS_DECLARE(x) int x; #define STATS_PRINT(x) printf(#var " = %d\n", x); 

Then:

STATS_FOREACH(STATS_DECLARE) // ... STATS_FOREACH(STATS_PRINT) 

This does not require the redefinition of macros and can make the code easier to understand and maintain.

Stringifying enumerations

C doesn't have introspection, but X macros can be used with preprocessor token stringification to let an enum print its own name, without manual updates to the stringification function.

#define SEASONS_FOREACH(X) \  X(winter) \  X(spring) \  X(summer) \  X(autumn) #define SEASONS_ENUM(x) x, enum season {  SEASONS_FOREACH(SEASONS_ENUM) }; #define SEASONS_CASE(x) \  case x: \  return #x; char *season_to_string(enum season value) {  switch (value) {  SEASONS_FOREACH(SEASONS_CASE)  } } 

This is expanded by the preprocessor to:

enum season {  winter,  spring,  summer,  autumn, }; char *season_to_string(enum season value) {  switch (value) {  case winter:  return "winter";  case spring:  return "spring";  case summer:  return "summer";  case autumn:  return "autumn";  } } 

season and season_to_string remain in sync as only the macro has to be updated, avoiding bugs related to one half being implemented but not the other.

Serialization

Once the X macro has been set up, the component macros can be redefined to generate, for instance, accessor and/or mutator functions. Structure serializing and deserializing are also commonly done.

Here is an example of an X macro that establishes a struct and automatically creates serialize/deserialize functions.

For simplicity, this example doesn't account for endianness or buffer overflows.

In star.def:

X(x, int) X(y, int) X(z, int) X(radius, double) #undef X 

star.def uses a variant of the separate file technique for writing X macros, where the #undef X that would be after every #include "star.def" in the including file is instead placed inside the list file, saving a line on each use.

In star_table.c:

struct shape { #define X(member, type) type member; #include "star.def" }; void serialize_star(struct shape *star, unsigned char *buffer) { #define X(member, type) \  memcpy(buffer, &(star->member), sizeof(star->member)); \  buffer += sizeof(star->member); #include "star.def" } void deserialize_star(struct shape *star, unsigned char *buffer) { #define X(member, type) \  memcpy(&(star->member), buffer, sizeof(star->member)); \  buffer += sizeof(star->member); #include "star.def" } 

Handlers for individual data types may be created and accessed using token concatenation (##) and quoting (#) operators. For example, the following might be added to the above code:

#define print_int(val) printf("%d", val) #define print_double(val) printf("%g", val) void print_star(struct shape *star) {  // print_##type will be replaced with print_int or print_double #define X(member, type) \  printf("%s: ", #member); \  print_##type(star->member); \  printf("\n"); #include "star.def" } 

This technique was reported by Lars Wirzenius in a web page dated January 17, 2000, in which he gives credit to Kenneth Oksanen for "refining and developing" the technique prior to 1997. The other references describe it as a method from at least a decade before the turn of the century.