The AM / FM category is a bit strange for effects - true, you can get some interesting effects by modulating the amplitude of an input signal ; but what does "frequency modulation" mean for a complex input signal for which you don't even have an accurate frequency representation? You could very well say that every effect is an amplitude modulation, by the $\frac{out(t)}{in(t)}$ signal, but this is not going to help you implement it!
There is no magic single-purpose engine at the heart of all the audio effects provided by music production software. But if you look at the source code of a large suite of audio effects from a DSP developer, here are some "building" blocks"building blocks" that will have been factored out in their own classes and which are shared by many different effects. I don't imply that this forms a good basis to categorize effecteffects ; but from an implementation point of view, those blocks are essential:
- Biquad filters.
- Fractional delay lines.
- Convolution engine, for fast convolution of an input signal with arbitrary-sized impulse responses.
- Waveshaper (application of a non-linear function to the input signal in the time-domaine).
- Synthesizer building blocks: oscillators, LFOs, ADSR envelopes.
- Signal detectors: envelope follower, f0 extractor.
With those blocks you could build:
- Synth-like filters or wah-wah: one or two biquads
- Auto-wah: envelope follower + envelope + biquad
- Flanger / Phaser: LFO + delay line
- Chorus: LFO + delay lines
- Algorithmic reverb: delay lines (array of parallel and serial comb filters)
- Convolution reverb: convolution
- Spatializer: convolution (with HRTFs impulse responses)
- Guitar amp simulation: convolution, waveshappers
- Distortion, Fuzz: gain + waveshapper
- Bitcrusher (quantizer): a particular case of waveshapper
- Ring modulator with a sine carrier: oscillator
- Noise Gate and other dynamics effects: envelope follower
- Tape simulation: convolution, waveshapper, envelope follower (for the dynamics processing)
- EQ: biquads
- Tremolo: LFO
- Leslie emulation: convolution + LFO
- Vocoder: biquads + envelope follower
While some of these effects are just a matter of patching the different blocks (a flanger truly is a LFO modulating a delay line), some other effects need more complex glue between the parts, that is specific enough not to be factored into a building block of its own.
This is, overall, an interesting set of effects that will cover a lot of ground for music production, but it's not exhaustive and there are indeed some effects which do not fit this framework... Some examples:
- Dirty sample rate reducer (in a bitcrusher): true, it's a multiplication by a dirac comb and then a convolution by a rectangular window... but it's easier to write it as something that duplicate the value of one sample over the N following samples instead of patching an amplitude modulation and a convolution!
- Pitch alteration effects (pitch-shifting, auto-tune) do not fit this framework well. They need a more complex representation of the audio signal to be performed (phase vocoder for frequency-domain methods ; pitch detection and pitch marks for time-domain methods).
I suggest you to play with "modular style" software tools like Pd, Reaktor, Plogue, SyntheEdit... - and try implement effects from the basic building blocks they provide you.