The manual page for userdel describes the -r option, which would remove the home-directory at the time you removed the user. If you did not do it then, you could find files with no known user with the find option -nouser, e.g.,
find /home -nouser -delete
though you might want to verify the list before actually deleting the files:
find /home -nouser -ls
The -delete option removes files. Removing directories would require a different command, e.g.,
find /home -type d -nouser -exec rmdir {} \;
(although some pathological user may have created a directory with embedded blanks, requiring more work).
By the way, /home is a convention, and you may have created your user's home-directories in other places.