The answer is if he is allowed to take an infinite number of steps the probability of him stepping off the cliff eventually is 1 (certain) for every value of p>0
EndTo demonstrate this, if you could continue flipping a coin infinitely it would not matter how many times your flipped it, at some point (because it is certain to happen if you could do it infinitely) you would flip a consecutive number of discussioneither tails or heads greater than the total number of flips you have already had. Therefore no matter whether the tails or the heads represent a 'step to the left' the man would always end up stepping over the line of the cliff that he started next to.