I have a section where its entries have a matrix field (matrixField). In the first matrix block there is a date field (dateField). I want to get my entries ordered by that date field. Is this possible? Pseudocode:
{% for entry in craft.entries.section('dates').order(matrixField.first().dateField).limit(5) %} ... Your example is referencing a field that Twig doesn't know about yet. Also, you usually place the fields you are ordering entries with in quotes (ex: order('dateField')).
Matrix blocks are elements (like so many things in Craft), and thus are unable to be used for ordering entries. THe reason is that the variable in the returned model is not the actual blocks, but a Criteria Model for retrieving those blocks. It's a reference, if you will. I suggest you create a separate date field on the entry for ordering.
My SuperSort plugin is built for use cases exactly such as this. It allows you to render each object in your array using a Twig object template and sort the array based on the rendered value.
You may be able to do the sorting with a bit of manual work. What you would need to do is to convert your ECM to a DBCommand and then manually join the matrix-field table that corresponds to the field you want to order by.
See Order two entry-types by different date fields for use in one combined stream for the approach I have taken to combine two entry types in a query, where one of them gets ordered by its matrix-field.
Used super sort plugin and got away with this:
{% for item in entries|supersort('sortAs', '{dateSet.first.startDate|date("c")}') %} 
Variable "matrixField" does not exist. The Craft docs say that "Possible values include 'title', 'id', ..., as well as any textual custom field handles." A matrix field is not a textual field and I do not think that the expression inside the order parameter gets evaluated.