There are, in fact, two folders whose names differ only in case. Git is case sensitive, since Unix systems are case sensitive and because it's impossible to correctly fold case in a locale-insensitive way.
You added two such directories and Git tracked them both, but you're on Windows, and by default, Windows ignores case, and so it's only possible to put one of them in the working tree. This is a limitation of Windows, not Git.
The Git FAQ contains an entry that deals with this scenario, which usually shows up as one file always being modified. What you shouldn't do is modify core.ignorecase, because Git sets that in the repository based on what it detects your file system is capable of. Changing the option doesn't change whether your file system is sufficiently capable, but it can make Git behave in bizarre ways which you probably weren't expecting on Windows.
core.ignorecasesetting. Rule for Git experts: almost never change thecore.ignorecasesetting. :-) Changing the setting lies to Git, making it misbehave. The misbehavior is predictable, and in some rare cases, useful, but it's not a good idea generally. The double rename trick you used works fine everywhere.