You want faster? You have to make a custom kernel. Ort you can just one that somebody made.
Basically all the junk is ripped out and you are left with basiscs. THen you add what you need.
Look at this site for a Raspberry Mini IMage
Also one of the first ones I ever saw is Damn Small Linux but I don't know if its compatible with ARM.
Here is a list of OTHER distributions that you might want to experiment with.