I personally use Dapple for writing Solidity unit tests. It has the advantage of not requiring a Javascript testing framework layer, so you can stay entirely in Solidity. Plus I can use it to test contracts that use imports, which I'm not sure is possible in chriseth's browser-based Solidity.
Edit: The accepted answer is definitely a better option for someone just looking to get a taste of the language. I do still hold that Dapple's a better option for people like myself who like to stay on the command line and who might want to knock out a quick test to shine some light into some of the language's darker corners. (You may easily underestimate my preference for the command line. It's a little ridiculous and extends to knocking out short scripts so I can check websites without having to switch to a browser.)
That said, since answering this question I've found a tool called Solidity REPL which does exactly what the name implies: gives you a Solidity REPL on the command line. For futzing about with the language and doing quick sanity checks, as one might with something like Python, I recommend that tool over Dapple.