What are Overlays?
First of all, they are elisp objects. This will be relevant later.
As you said yourself, they represent layers that apply on top of regions of the buffer. These layers have text properties, just like the actual text in the buffer. Any regular property that an overlay has, applies to the text below it. However, there are some properties that are special to overlays (they don’t do anything if applied on text).
Why are they Useful?
For the two reasons I hinted at above:
They are objects
This means you can store them in lists and handle them easily. You can change their properties without having to keep track of where they are right now. Even better, you can use them to keep track of where things are in the buffer.
This answer by Stefan is a good example of overlays being used to keep track of buffer regions. Here’s a short snippet of it.
(let ((ol (make-overlay beg end))) (overlay-put ol 'evaporate t) (overlay-put ol 'my--auto-align-regexp regexp) (push ol my--auto-align-overlays))
It uses overlays to record which regions are to be aligned, and which regexp to use on each of them. my--auto-align-overlays is
a list where the overlays are stored, and they can be conveniently accessed by looking through this list.
In contrast, if we were using text properties for this (which is possible) we would have no easy way of accessing them. You need to parse the buffer to find text properties.
They are slightly more powerful
There are a few text-properties which only have an effect on overlays. You can find the entire list on the manual. It’s too large to include here, but here’s a short highlight.
before-string
This property's value is a string to add to the display at the beginning of the overlay. The string does not appear in the buffer in any sense—only on the screen.
line-prefix
This property specifies a display spec to prepend to each non-continuation line at display-time. See Truncation.
wrap-prefix
This property specifies a display spec to prepend to each continuation line at display-time. See Truncation.
In particular, the before-string property allows you to affect the display of the buffer even with a 0 width overlay. Which is something you can’t do with text property. Everything you do with text properties needs to either go over an existing text (which might hide this text) or go over a new string of text you insert (which alters the real contents of the buffer).
Here’s an example snippet about that. Evaluate it on a temporary buffer.
(overlay-put (make-overlay (point) (point)) 'before-string "Hi there!")
When are they Bad?
Overlays are much more demanding (processing wise) than text properties. Some relevant operations (such as text insertion, if I’m not mistaken) take time proportional to the number of overlays in a buffer. For that reason, they are unsuitable when you need a large number of them. In this case you should resign to text properties.
i text properties, of course.