I'll limit my answer to neural networks.
Is it reasonable for someone with shallow knowledge in math but programming skills to dive into neural network/machine learning studies?
It's reasonable and possible. Here are few reasons to support this conclusion:
Neural networks are inspired by the functioning of our brains. Therefore lots of concepts are familiar and easy to understand: neurons, connections, activation etc. This makes the introduction to neural networks smooth and exciting, and doesn't require any math.
The basic operation of a neural network, regardless of its size, is easy to understand: forward passing, signals flowing from one level to another, neuron activation etc. Not too much math required here: wighted sums, and non-linear functions such as sigmoid.
The math underlying some of the most fundamental algorithms for training neural networks (e.g., back propagation) is not complex: sums, logarithms, multiplications and divisions. And you calculate values that have concrete meaning: error cost, gradient etc. Some other ML techniques require calculation of intermediary values (often matrices) whose meaning is not that clear or intuitive.
It's often not the more sophisticated math but rather understanding and experience with the basics that let you dive into more advanced topics such as regularisation, pre-training, dropout etc.
To make things clear there're more complex network architectures and mathematically demanding algorithms for training neural networks. Also, although calculations involved in back propagation are simple its derivation is complex for someone who didn't study calculus. Still the difficulty of derivation or existence of more complex algorithms doesn't mean that studying neural networks is unreasonable. These issues won't prevent you from building good understanding and making practical use of neural networks.
Is it possible to build interesting projects in this area, like those playing atari, using only high-level tools?
In the recent years the interest in neural networks (deep learning especially) is spiking. This resulted in the creation of many good tools and libraries as you observed, some of which are coming from factories such as Google, Microsoft or NVidia. No doubt the quality of these is sufficient to create interesting projects. What might prove more challenging is getting the right amount of quality data to train your network (given such data is not currently available).