I would segregate the persistent data to DTO's that are consumed and produced by full blown long lived objects that were taught what their handlers were when they were constructed.

Done this way you don't have to create hierarchies or libraries. Persistent data comes and goes. The objects that know how to deal with that data send messages to other objects telling them what they need done. 

Each long lived object should be no more complex than is needed to handle a single idea.

I've used this pattern successfully many times. My favorite was a chess AI me and my teammate created (that, incidentally, won the tournament). Each kind of game piece had one object. Even though there are 16 pawns on the board I had only one pawn object. And it didn't know where it was on the board until you told it. I stored many references to it on the board. It's only job was to figure out what it's legal moves were.

I know this flies in the face of the style of "modeling the world as objects" but I've found true object oriented programing isn't done that way at all. Sometimes I create objects for things that don't even exist in the real world.

Don't be afraid to write code driven by your needs rather than some prescribed style. Most people giving you advice have no idea what you're up to.