The whole point of interrupts is that they can (and do) happen all the time, and are designed to have zero impact on any code that happens to be running when they occur. All the registers are saved, and depending on the CPU architecture, a completely different register set may be swapped in, the interrupt does its thing, and then the original registers are restored and the code continues to run as normal.
Problems can occur when the interrupt service routine itself tries to access memory that is being accessed by the running, interrupted code. Even more subtle errors can occur when a time-critical I/O process is interrupted. These problems are commonplace with older, simpler, less secure architectures where there may be little separation between "user" and "supervisor/kernel" mode code.
This kind of problem can be hard to identify, and often difficult to reproduce, but once identified they're often fairly trivial to fix using defensive programming, mutexes/semaphores or simply by disabling interrupts in critical sections of code.
The general class of problems has been studied extensively, and modern multi-core CPUs and even multi-tasking operating systems would not be possible if multiple solutions were not already tried and tested.