You need to distribute a few .dlls alongside the executable, in the same directory.
ntldd -R my_program.exe will give you a list of DLLs. Out of those, ignore anything that's not in your compiler's bin directory (note that the paths printed by ntldd can be wrong if you have those DLLs in other directories in PATH; so ignore the paths and compare just the filenames).
On my MinGW distribution, this gives me libgcc??.dll, libstdc++??.dll, and libwinpthread??.dll.
#includes are not "libraries", see Is iostream a header or a library. They don't have one-to-one correspondence with the DLLs.
Out of the three DLLs above, libstdc++ is the entirety of the C++ standard library, libwinpthread is the low-level threading library libstdc++ uses under the hood, libgcc contains low-level utilities used by GCC, and the C standard library mostly resides in system-provided DLLs (either msvcrt.dll or ucrtbase.dll, depending on MinGW flavor; you don't need to ship those).
A static linking (via the -static flag) is also an option, which will give you a single self-contained .exe.
But eventually you'll run into the libraries with licenses that discourage static linking (LGPL forces you to share source if you link statically), and/or build systems that don't produce static libraries out of the box (e.g. OpenAL; which uses LGPL), so better not get used to it. Also allowing the user to update the libraries by themselves is a good idea.