Having a *"Map object that keeps track of all the cells to which it can travel"* looks to me like an example of premature optimization (which, as your surely know, D. Knuth called "the root of all evil").

Instead of remembering all those cells for each piece "just in case it could become slow", why not give a piece a method with the board as parameter, which returns the related `List<Cell>` by *calculating it* when the method is called? This way, no notifications or observer pattern or "32 listeners" are needed any more.

If it turns out later that this method is called quite often for the same, unchanged board so it becomes a performance bottleneck, you can still cache its result and invalidate the cache for all pieces whenever the board is changed.