The TOTP standard considers this.
It RECCOMENDs to allow for 1 timestep of delay (30 seconds)
It states that a verifier "SHOULD typically set a policy for an acceptable OTP transmission delay window" So even codes that look to have expired on your client may be valid to the server.
In regards to avoiding the issue, the HOTP algorithm (essentially TOTP but with a counter not a timestamp) solves this specific issue but introduces others, such as desynchronisation.
