UPDATE: As of 2025, the accepted answer is out-of-date. Here's a more up-to-date answer.

Breaking it down, there's two answers: historical and modern.

1. /* vs /usr/*: today, /bin and /sbin link to their /usr/bin and /usr/sbin siblings. In the past, /bin and /sbin were separate because of files needed at startup; /bin and /sbin included files needed early in the startup process (as /usr was sometimes mounted later on).
2. */bin vs */sbin: today, the sbin folders are usually just links to bin. Historically, the s in /sbin stood for system, and referred to core system programs, which are usually called by the system rather than the user (think drivers, not Chrome).
3. /usr/* vs /usr/local/*: /usr/* programs are installed and owned by the distribution's package manager (e.g. apt or pacman). /usr/local/* is used for user-owned programs, e.g. locally compiled packages, that aren't owned by your package manager.
4. /opt is mostly for packages that don't play by Unix filepath conventions.

UPDATE: As of 2025, the accepted answer is out-of-date. Here's a more up-to-date answer.

Breaking it down, there's two answers: historical and modern.

1. /* vs /usr/*: today, /bin and /sbin typically link to (or from) their /usr/bin and /usr/sbin siblings, effectively making them synonyms. In the past, /bin and /sbin were separate because of files needed at startup; /bin and /sbin included files needed early in the startup process (as /usr was sometimes mounted later on).
2. */bin vs */sbin: today, the sbin folders are usually just links to bin. Historically, the s in /sbin stood for system, and referred to core system programs, which are usually called by the system rather than the user (think drivers, not Chrome).
3. /usr/* vs /usr/local/*: /usr/* programs are installed and owned by the distribution's package manager (e.g. apt or pacman). /usr/local/* is used for user-owned programs, e.g. locally compiled packages, that aren't owned by your package manager.
4. /opt is mostly for packages that don't play by Unix filepath conventions.