In the big scheme of things it's all about finding the right balance in life.
What's important is whether you enjoy programming and whether you keep learning no matter what level you're at. Whether or not you program outside work does not automatically make you a "good" or "bad" programmer.
To give a personal perspective, I've been programming for about 25 years (first as a kid, then professionally). I absolutely love it.
However, I almost never program outside work. This has to do with several things:
- I am very lucky in that at work I get to do what I love, every day, all day long.
- Quite simply, there are other things in life and there are only so many hours in the day.