You have two good answers already; let me provide a more succinct third. Let's start by re-stating the question:
how does limiting a type to an output (or input) position achieve preservation (or reversal) of the assignment compatibility relation?
The key insight is to think about a simple function. Let's suppose every mammal has a friend who is also a mammal.
static Dictionary<Mammal, Mammal> friends = whatever; static Mammal F (Mammal input) => friends[input]; The question is: how can we legally change (vary!) the type signature of F without changing the body or the declaration of friends? (Ignoring any existing callers of F; we're only concerned with the body of F staying legal.)
static Mammal F (Animal input) => friends[input]; // WRONG, we could pass a Turtle static Mammal F (Giraffe input) => friends[input]; // RIGHT, this is fine static Animal F (Mammal input) => friends[input]; // RIGHT, this is fine static Giraffe F (Mammal input) => friends[input]; // WRONG, it might return a Tiger The conclusion is that the body of a Func<Mammal, Mammal> could always be used in a context where, say, a Func<Giraffe, Animal> is required but not in a context where Func<Animal, Giraffe> is required.
That's why there is a connection between covariance and outputs, and contravariance and inputs. It follows from the basic facts about how function bodies work.