Yes OO can often be used to model real world entities.

Even in my business layer I've got classes like observers, managers, factories, etc. which aren't real world objects. 

Do not confuse object oriented development with design patterns. OO analysis and design is a means to approach programming maintainable code. Coupled with an OO language programmers are given the power to create re-useable code through the pillars of OO: encapsulation, polymorphism, and inheritance.

To encapsulate an entity we can model that entity after its real world counterpart. For example, if we have a guitar then a guitar class encapsulates the behaviors and properties of a real world guitar. We can further abstract the guitar as, say, an IInventoryItem to take advantage of the potential for code re-use through polymorphism and inheritance.

On the other hand, we may find that a factory of guitars could assist us in maintening a set of different types of guitars. This isn't because of OO. Rather, a factory is a design pattern that has stood the test of time as a proven means of successfully creating maintenanble code for such a purpose. In other words, we programmers are often solving similar problems. So we came up with a common solution for solving them (don't reinvent the wheel).

That does not mean OO has to model the real world, nor that it is always the most optimal solution to do so. Simply, that as a rule of thumb "OO modeling the real world" makes perfect sense.

Yes, OO can often be used to model real world entities.

Even in my business layer I've got classes like observers, managers, factories, etc. which aren't real world objects.

Do not confuse object oriented development with design patterns. OO analysis and design is a means to approach programming maintainable code. Coupled with an OO language, programmers are given the power to create re-usable code through the pillars of OO: encapsulation, polymorphism, and inheritance.

To encapsulate an entity we can model that entity after its real world counterpart. For example, if we have a guitar then a guitar class encapsulates the behaviors and properties of a real world guitar. We can further abstract the guitar as, say, an IInventoryItem to take advantage of the potential for code re-use through polymorphism and inheritance.

On the other hand, we may find that a factory of guitars could assist us in maintening a set of different types of guitars. This isn't because of OO. Rather, a factory is a design pattern that has stood the test of time as a proven means of successfully creating maintenanble code for such a purpose. In other words, we programmers are often solving similar problems. So we came up with a common solution for solving them (don't reinvent the wheel).

That does not mean OO has to model the real world, nor that it is always the most optimal solution to do so. Simply, that as a rule of thumb “OO modeling the real world” makes perfect sense.