What you want to do for this kind of game, is not directly change the transform of the ball but change the velocity of the rigidbody. 

Set friction to 0 in the rigidbody of the ball and lock the movements of the panels. 

Have a script that detects if the ball collides with the panel, on collision delete the panel and let the physics response do the work. 

Set the physics material of the ball/walls/panels to an elastic collision. 

If you want to add some speed when the ball hits a panel/player, increment/alter the velocity or apply a force to the rigidbody.

The player should not be static. Static is used to optimize non moving GameObjects. The player should have a its rigidbody x and y positions locked and moved with a script on user input. Lock the rigidbody under constraints in the inspector.

Hope this helps!

[Here is a Tutorial that might help!][1]


 [1]: https://www.google.ca/url?sa=t&rct=j&q=&esrc=s&source=video&cd=1&ved=0CBwQtwIwAA&url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DRrdzLwZYz8Q&ei=k_j4VMPJDoiXgwTXzoPQCw&usg=AFQjCNFGMHKc9Pjf14BWyebIM4-kXUC72g&sig2=UwxcBWFmNubp192QTgyjoA&bvm=bv.87611401,d.eXY&cad=rja