You ask "Whats the mistake I make ?" . Your following line is where I think you went wrong:
What I mean to say is that the subject Class must know the observer object to add him to the list of objects to notify.
ONLY THE INTERFACE resides with the subject class. The subject class ONLY deals with the interface. Anybody can come along and implement the interface.