How do mipmapping, anti-aliasing and anisotropic filtering contribute to rendering quality?

Firstly, texture filtering and anti aliasing are two different ways of improving image quality. You can apply them both independently.

Anti aliasing is primarily done to smooth the joins where two separate triangles overlap. Depending on how it's done it can also benefit texturing quality indirectly too.

Texture filtering is done to improve the image quality inside each triangle.

Mip mapping is done to make texture filtering efficient. To take it to extremes let's say you had a GPU that could filter textures perfectly without mip maps. If you had a square textured with a 1024x1024 texture that occupied a single pixel on screen, that theoretical GPU would have to read all of the 1,048,576 pixels in the texture to work out the colour of that one pixel on screen. With a mip map it can read one pixel from the smallest mip level to do the same thing and get the same result.

Of course in reality without mip maps a GPU will actually only look at a very small number of pixels in the texture it's sampling (four of them with bilinear filtering, one with point filtering). This generally causes the result to be very noisy, and also gives a performance hit because of cache misses as those pixels are scattered all over the texture. The only places where it can be sensible not to use mip maps are for things like 2D sprites where you don't scale the image significantly.

The cost of adding a mip map to a texture is that it gets about 33% bigger, which is a relatively low memory overhead given the benefits.

Anisotropic filtering kicks in if instead of one pixel in that previous example it was say a 1x8 pixel square. What happens if you use the 1x1 mip level is that the result has no detail and looks blurry. However the 8x8 mip level has 64 pixels in it, which need to map to 8 pixels on screen. With anisotropic filtering the GPU will read some extra pixels from a bigger mip level to make the result less blurry. Reading those extra pixels has a performance cost compared to bilinear or trilinear filtering.

In a more realistic scene the anisotropic filtering makes things that aren't square on to the camera like floors less blurry.

Image is from http://en.wikipedia.org/wiki/Anisotropic_filtering

In an ideal world, we wouldn't want them. We'd just use the same high quality textures far away and not care about wasted resources.

Not really. If we forget about wasted resources and use fast enough machines, simple sampling of the textures, with e.g. bilinear filtering, will result in what is known as aliasing. In particular, this will look like noise in the areas where the texture is effectively downsampled. See e.g. faraway part of the pavement in the following scene (click to see in 100% scale to avoid blurring):

This noise will also flicker badly on slightest motion of the camera.

In a really ideal world we would fight this problem of texture aliasing (and all other aliasing problems, e.g. jagged edges of polygons) with supersampling applied to the full scene. With this technique applied, the scene above would look like this:

Now, our world is not really ideal. Moreover, it's not even slightly ideal :) In particular, supersampling is very demanding on GPU. It basically means rendering the same scene N times, where N is the number of samples per pixel (usually N=16 is a good enough figure for polygon edges, not so good for textures).

Mip mapping is a technique that aims to approximate the ideal result of supersampling by downsampling the textures in steps, reducing their size 2× at a time, and then sampling from the closest mip level (or using weighted average of two closest, via linear interpolation). When generating the mip map, each downsampling step is done in a high-quality way, avoiding aliasing. This reduces noise and generally works satisfactorily for objects whose sides don't differ too much in level of detail.

But in computer games and some other kinds of graphics software, inclined surfaces like e.g. roads appear quite often. The result is that, although in one dimension the texture is squeezed, its other dimension's squeezing coefficient is much smaller. The result is that the above scene gets approximated poorly. The noise does get removed, but blurriness is excessive:

Anisotropic filtering is a modification of mip mapping, where sizes of each mip level get applied separately to width and to height of the image, yielding combinations of different widths and heights in the resulting mip map. This takes considerably more space than normal mip map (up to 4× the original texture), but the result is a drastic increase of image quality. The texture rendering becomes comparable to the supersampled result: