JTAG was originally an interface used for testing PCBs after assembly. The IO lines on the chips could be controlled and read via the JTAG ports allowing a board test sequence to be performed. Later on it also started to be used as a programming and debug interface. What functionality is available over JTAG will depend on the particular IC you are dealing with.
One advantage of JTAG over most other programming interfaces is that devices can be daisy chained. So one JTAG port can serve to program/debug/test multiple devices (though I would advise designing your systems so that different brands of chip can be split on to different chains, while the low level JTAG protocol is pretty standardised unfortunately the tools you need to program/debug the chips are not).