The abstraction is sort of the point. Using the 'man day' as a measurement has a number of pitfalls, including:

1. If the team isn't familiar with the tech they are going to be using, then it can be really hard to give real-time estimates of how long a task might take. They are much more likely to be able to give good relative estimates - e.g. "task A will probably take twice as long as task B".
2. Different people work at different rates! If you use 'man days' you pretty much have to change the time estimate when a task is passed from one developer to another. Who defines how much work constitutes a 'man day' anyway?

If you want to estimate man-days it's a simple calculation:

user points in story / average user points per developer per day = estimated man days