You have two good answers already; let me provide a more succinct third. Let's start by re-stating the question:
how does limiting a type to an output (or input) position achieve preservation (or reversal) of the assignment compatibility relation?
The key insight is to think about a simple function. Let's suppose every mammal has a friend who is also a mammal.
static Dictionary<Mammal, Mammal> friends = whatever; static Mammal F (Mammal input) => friends[input]; The question is: how can we legally change (vary!) the type signature of F without changing the body or the declaration of friends? (Ignoring any existing callers of F; we're only concerned with the body of F staying legal.)
static Mammal F (Animal input) => friends[input]; // WRONG, we could pass a Turtle static Mammal F (Giraffe input) => friends[input]; // RIGHT, this is fine static Animal F (Mammal input) => friends[input]; // RIGHT, this is fine static Giraffe F (Mammal input) => friends[input]; // WRONG, it might return a Tiger The conclusion is that a Func<Mammal, Mammal> such as F could always be used in a context where, say, a Func<Giraffe, Animal> is required but not in a context where Func<Animal, Giraffe> is required.
That's why there is a connection between covariance and outputs, and contravariance and inputs. It follows from the basic facts about how function bodies work. Making an output type declaration bigger or an input type declaration smaller preserves the correctness of the body.