You cannot chroot into different architecture. By chrooting, you are executing the binaries (from the chroot) on your architecture. Executing ARM binaries on x86 (and x86_64 in that matter) would lead to "Exec format error".
If you want to run binaries from different architecture you will need an Emulator. Qemu is a good candidate for this, but you will need to learn how to use it. This would involve creating RootFS and compiling a kernel for ARM. You will need a toolchain for compiling ARM binaries (and kernel) perhaps. One thing is for sure: Forget the chroot method, you cannot run binaries compiled for ARM on x86 (x86_64).
Edit: After the small talk with @UrichDangel, I realized, it should be possible to enter the chroot environment with qemu-user programs (qemu-arm in this case). Chroot should be executing qemu-arm compiled for your host architecture, then the qemu-arm can execute your /bin/sh (compiled for arm).
