Akwardness or hoop-jumping to unit test such a "singleton" is one potential downside in addition to serialization.
Contrast this with unit testing a true (i.e. instantiable) singleton.
Ultimately, a singleton guarantees a single instance of a class, whereas a static class is not instantiable as @JStevenPerry points out (and I expect you already understand): the two are simply not the same although they can in many ways be employed similarly.