If you always want to complete directory names only for ls, you can put this in your .zshrc:
compdef _dirs ls
You can do fancier stuff with the “new” completion system (initialized by compinit) by playing with styles. Depending on your options, you may need to unalias ls. Then, to only ever complete directories on the ls command line:
zstyle ':completion:*:ls:*' file-patterns '*(/):directories'
You can complete only directories by default, but complete any file name if no directory matches:
zstyle ':completion:*:ls:*' file-patterns '%p:globbed-files' '*(/):directories'
You can also define a key binding to complete only directories, which you can then use anywhere.
zle -C complete_dirs .complete-word _dirs bindkey '^X/' complete_dirs