You could use `sed`. The following would remove lines that are 3 characters long or smaller:

 sed -r '/^.{,3}$/d' filename

In order to save the changes to the file in-place, supply the `-i` option.