Dynamically subclass an Enum base class

Question

I've set up a metaclass and base class pair for creating the line specifications of several different file types I have to parse.

I have decided to go with using enumerations because many of the individual parts of the different lines in the same file often have the same name. Enums make it easy to tell them apart. Additionally, the specification is rigid and there will be no need to add more members, or extend the line specifications later.

The specification classes work as expected. However, I am having some trouble dynamically creating them:

>>> C1 = LineMakerMeta('C1', (LineMakerBase,), dict(a = 0)) AttributeError: 'dict' object has no attribute '_member_names'

Is there a way around this? The example below works just fine:

class A1(LineMakerBase): Mode = 0, dict(fill=' ', align='>', type='s') Level = 8, dict(fill=' ', align='>', type='d') Method = 10, dict(fill=' ', align='>', type='d') _dummy = 20 # so that Method has a known length A1.format(**dict(Mode='DESIGN', Level=3, Method=1)) # produces ' DESIGN 3 1'

The metaclass is based on enum.EnumMeta, and looks like this:

import enum class LineMakerMeta(enum.EnumMeta): "Metaclass to produce formattable LineMaker child classes." def _iter_format(cls): "Iteratively generate formatters for the class members." for member in cls: yield member.formatter def __str__(cls): "Returns string line with all default values." return cls.format() def format(cls, **kwargs): "Create formatted version of the line populated by the kwargs members." # build resulting string by iterating through members result = '' for member in cls: # determine value to be injected into member try: try: value = kwargs[member] except KeyError: value = kwargs[member.name] except KeyError: value = member.default value_str = member.populate(value) result = result + value_str return result

And the base class is as follows:

class LineMakerBase(enum.Enum, metaclass=LineMakerMeta): """A base class for creating Enum subclasses used for populating lines of a file. Usage: class LineMaker(LineMakerBase): a = 0, dict(align='>', fill=' ', type='f'), 3.14 b = 10, dict(align='>', fill=' ', type='d'), 1 b = 15, dict(align='>', fill=' ', type='s'), 'foo' # ^-start ^---spec dictionary ^--default """ def __init__(member, start, spec={}, default=None): member.start = start member.spec = spec if default is not None: member.default = default else: # assume value is numerical for all provided types other than 's' (string) default_or_set_type = member.spec.get('type','s') default = {'s': ''}.get(default_or_set_type, 0) member.default = default @property def formatter(member): """Produces a formatter in form of '{0:<format>}' based on the member.spec dictionary. The member.spec dictionary makes use of these keys ONLY (see the string.format docs): fill align sign width grouping_option precision type""" try: # get cached value return '{{0:{}}}'.format(member._formatter) except AttributeError: # add width to format spec if not there member.spec.setdefault('width', member.length if member.length != 0 else '') # build formatter using the available parts in the member.spec dictionary # any missing parts will simply not be present in the formatter formatter = '' for part in 'fill align sign width grouping_option precision type'.split(): try: spec_value = member.spec[part] except KeyError: # missing part continue else: # add part sub_formatter = '{!s}'.format(spec_value) formatter = formatter + sub_formatter member._formatter = formatter return '{{0:{}}}'.format(formatter) def populate(member, value=None): "Injects the value into the member's formatter and returns the formatted string." formatter = member.formatter if value is not None: value_str = formatter.format(value) else: value_str = formatter.format(member.default) if len(value_str) > len(member) and len(member) != 0: raise ValueError( 'Length of object string {} ({}) exceeds available' ' field length for {} ({}).' .format(value_str, len(value_str), member.name, len(member))) return value_str @property def length(member): return len(member) def __len__(member): """Returns the length of the member field. The last member has no length. Length are based on simple subtraction of starting positions.""" # get cached value try: return member._length # calculate member length except AttributeError: # compare by member values because member could be an alias members = list(type(member)) try: next_index = next( i+1 for i,m in enumerate(type(member)) if m.value == member.value ) except StopIteration: raise TypeError( 'The member value {} was not located in the {}.' .format(member.value, type(member).__name__) ) try: next_member = members[next_index] except IndexError: # last member defaults to no length length = 0 else: length = next_member.start - member.start member._length = length return length

Paul Cornelius · Accepted Answer · 2017-05-16 23:55:30Z

This line:

C1 = enum.EnumMeta('C1', (), dict(a = 0))

fails with exactly the same error message. The __new__ method of EnumMeta expects an instance of enum._EnumDict as its last argument. _EnumDict is a subclass of dict and provides an instance variable named _member_names, which of course a regular dict doesn't have. When you go through the standard mechanism of enum creation, this all happens correctly behind the scenes. That's why your other example works just fine.

This line:

C1 = enum.EnumMeta('C1', (), enum._EnumDict())

runs with no error. Unfortunately, the constructor of _EnumDict is defined as taking no arguments, so you can't initialize it with keywords as you apparently want to do.

In the implementation of enum that's backported to Python3.3, the following block of code appears in the constructor of EnumMeta. You could do something similar in your LineMakerMeta class:

def __new__(metacls, cls, bases, classdict): if type(classdict) is dict: original_dict = classdict classdict = _EnumDict() for k, v in original_dict.items(): classdict[k] = v

In the official implementation, in Python3.5, the if statement and the subsequent block of code is gone for some reason. Therefore classdict must be an honest-to-god _EnumDict, and I don't see why this was done. In any case the implementation of Enum is extremely complicated and handles a lot of corner cases.

I realize this is not a cut-and-dried answer to your question but I hope it will point you to a solution.

That block of code is gone because it is Python 2.x specific.
Or, to be more accurate, that code block was only added to the backport because the backport supports Python 2.x and under normal circumstances the stdlib EnumMeta will never see a dict-type clsdict.

Community · Accepted Answer · 2017-05-23 11:47:15Z

Create your LineMakerBase class, and then use it like so:

C1 = LineMakerBase('C1', dict(a=0))

The metaclass was not meant to be used the way you are trying to use it. Check out this answer for advice on when metaclass subclasses are needed.

Some suggestions for your code:

the double try/except in format seems clearer as:

 for member in cls: if member in kwargs: value = kwargs[member] elif member.name in kwargs: value = kwargs[member.name] else: value = member.default

this code:

# compare by member values because member could be an alias members = list(type(member))

would be clearer with list(member.__class__)
has a false comment: listing an Enum class will never include the aliases (unless you have overridden that part of EnumMeta)

instead of the complicated __len__ code you have now, and as long as you are subclassing EnumMeta you should extend __new__ to automatically calculate the lengths once:

# untested def __new__(metacls, cls, bases, clsdict): # let the main EnumMeta code do the heavy lifting enum_cls = super(LineMakerMeta, metacls).__new__(cls, bases, clsdict) # go through the members and calculate the lengths canonical_members = [ member for name, member in enum_cls.__members__.items() if name == member.name ] last_member = None for next_member in canonical_members: next_member.length = 0 if last_member is not None: last_member.length = next_member.start - last_member.start

Yeah- I created my own metaclass because I wanted to be able to change the __str__ of the child class object, and this was not possible with a decorator.
You can do that by putting the __str__ in the base class. You can do just about everything you need without metaclass hackery.
But the base class __str__ will not appear in the MRO for the child class, correct? I wanted to customize the str output for the class object, not the instance object.
@RickTeachey: Ah, I see. Yes, if you want the class to show up differently then you have to change the metaclass __str__.
Ah ok, I was hoping I was wrong. Additionally, I want to be able to write code like this: LineChild.format(**some_dict), so format needs to be in the class object MRO, not the member MRO.

Berislav Lopac · Accepted Answer · 2018-03-20 23:39:46Z

The simplest way to create Enum subclasses on the fly is using Enum itself:

>>> from enum import Enum >>> MyEnum = Enum('MyEnum', {'a': 0}) >>> MyEnum <enum 'MyEnum'> >>> MyEnum.a <MyEnum.a: 0> >>> type(MyEnum) <class 'enum.EnumMeta'>

As for your custom methods, it might be simpler if you used regular functions, precisely because Enum implementation is so special.

Collectives™ on Stack Overflow

Dynamically subclass an Enum base class

3 Answers 3

3 Comments

5 Comments

Comments

Linked

Hot Network Questions