In this note, I will argue that the jaggedness observed in rasterized images should not be referred to as aliasing.
First, let us agree on a definition: aliasing is the distortion that arises when overlapping copies of a signal's spectrum combine, causing high-frequency components to be misrepresented as lower-frequency ones.
This misrepresentation leads to artifacts such as beats or moiré patterns in images, as observed when textures are resampled at an insufficient frequency. Visually, this is clearly distinct from jaggedness, and I believe we can all agree on that — raising the question posed in the title.
Let us review what the Shannon-Nyquist sampling theorem prescribes:
- Given that the signal has a limited bandwidth, it must be sampled at a sufficiently high frequency (the Nyquist frequency).
- The appropriate reconstruction filter must be used — specifically, the sinc function, as given by the Whittaker–Shannon interpolation formula.
I maintain that aliasing occurs when condition (1) is violated, whereas "jaggedness" arises from the violation of (2).
In other words, jaggedness is not introduced during sampling but rather during reconstruction. When visualizing the signal on a raster display, we are forced to reconstruct it using a box filter (or zero-order hold) instead of the ideal sinc filter. This violation of (2) results in the jagged appearance.
I realize that this is not how "aliasing" is commonly interpreted in mainstream computer graphics. However, I can cite several authors who support this perspective.
The consequence of failing to filter the signal properly before sampling is known as "aliasing." Aliasing occurs when a lower frequency signal appears as an "alias" of a high frequency signal after sampling. Therefore highly periodic images of scenes involving, for example, picket fences or venetian blinds may appear, when sampled, to be made up of a few broad strips rather than many fine lines.
Reproducing the signal involves representing each sample in such a way that the reproduced signal has no frequencies higher than the original signal. This can be accomplished by representing each sample as a rectangular pulse and then low-pass filtering the resulting signal. In the two-dimensional case, the result of failing to filter the signal properly during reconstruction is known as "rastering." Rastering is an artifact of the structure of the displayed image.
[The Aliasing Problem in Computer-Generated Shaded Images Franklin C. Crow. Comm of the ACM 11:20, 1977]
Although aliasing may occur in the image-synthesis process, jaggedness is primarily due to pixel-structure artifacts that arise during reconstruction of a sampled signal by the display pixels. In fact, a completely “antialiased” image may still appear jagged, due to inexact reconstruction. Furthermore, few of the algorithms in the literature actually guarantee that the resultant sampled signal has no lower-frequency aliases. Hence, we prefer the term “filtering” rather than “antialiasing.
[Avi C. Naiman. Jagged Edges: When Is Filtering Needed? ACM Transactions on Graphics, Vol. 17, No. 4, October 1998.]
In summary, my point is that jaggedness is not an aliasing effect caused by sampling below the Nyquist frequency. Rather, it is an artifact of using a non-ideal reconstruction kernel.
I would appreciate your thoughts on this.
Edit: I am appending my reply here since I do not have enough reputation to comment.
Thanks a lot for this discussion. As often happens, everyone is right. Of course, one is free to call "aliasing" any artifact that arises in the sampling and reconstruction process, including truncation, quantization, wrong reconstruction function and even the discrete nature of the display.
But if we agree (and you might not, of course) that aliasing specifically refers to the effect where certain frequencies are incorrectly substituted for others (hence the name "alias": a false or assumed identity), occurring due to copies of the original spectrum overlapping because of sampling below the Nyquist rate, then, I find it difficult to relate this to "jaggedness", whereas it fits perfectly to moiré patterns in texture mapping.