How is game difficulty typically tested or balanced so that there's a steady progression of difficulty; how do I ensure that no level is too difficult or too easy for its point in the game?
Are play testers of various skills brought in, or a random sampling of people? What considerations are/should be taken into account during the actual level design process?
Two big helps in difficulty tuning.
Metrics
Tracking player data can go a long way to making difficult tuning as objective a process as possible. How long are people staying alive, where are the taking damage or getting killed, how many tries does a section of the level take. Getting good metric data can really help your team see where people are having issues in your game.
Kleenex Testers
The longer your QA staff plays the game the better they get at it, so it's good to have sessions where you grab new people who have never sat down with the game and have them go through it.
Watch someone play but don't talk to them. By not talking to them, you can see things them do things that will make you pull you hair (You: the solution is obvious. What's wrong with you? #$@!).
Instead of saying your thoughts aloud, write down what you want to say. Use this as your basis to guide your game difficulty.
Ensure that you change the people playtesting your game. We shipped a title for the original PSX that was far too difficult (far too tight time limits) as we'd had all the same playtesters testing the game over the development period so they knew the levels backwards.
Some games adjust the difficulty based on the skill of player during the game.
For example every time a player dies at a level - difficulty decreases. Every time he finishes - difficulty increases.
Still this needs some testing to get the decreasing and increasing right.
This method makes the scoring a little less accurate and harder to compare with others but the game more enjoyable as you won't get totally stuck.
an idea I've seen mentioned elsewhere is to sit pairs of people in front of the application[game in this case]. They're more likely to reveal their thought processes to someone else watching and the 2nd person may ask questions without leading them like you or another team member might.
First a quality assurance pass is needed to make sure the tasks are reasonable and completable by your internal QA.
Now do a sweep on initial difficulty and/or level times, rinse and repeat until diminishing returns mean you need to move on.
The next step is to use all the other internal staff you can grab for a few play-thoughs. Be sure to record as much as possible for later analysis. Sweep, iterate, repeat.
Families are a good next line if your on a budget, after that you need to bring in focus groups or outsource testing.
Remember do small chunks, iterate and repeat.
