The two very standard things you can do are (i) to assign to one class of the probability is greater than or equal to 0.5 (or whatever threshold is appropriate for your task) and the other class if the probability is less than 0.5; and (ii) have some zone of probability where the uncertainty is too great to make a decision on that basis, i.e. to have a "reject" option (for multi-class problems, reject if the difference in probability between the most probable and the second most probable class is below some cut-off value).
I have to say I disagree with those who argue against having a threshold. It depends on the needs of the application, it isn't a statistical issue. In some applications you have to make a decision, and the quality of that decision may be something we need to measure. In some applications it is acceptable to have a reject option (for instance it may be a screening test to triage cases sent for a more expensive evaluation) and some it isn't. In some applications, where perhaps the operational class frequencies or misclassification costs are unknown or are variable, in which case we are better off focussing on probability estimation in a way that is independent of the threshold (because we don't know the appropriate value of the threshold). Unfortunately there are cases where a probabilistic models give worse decisions (for a fixed threshold) than purely discriminative "hard" classifiers, such as the SVM, so we can't assume that probabilistic classifiers are a panacea - they aren't. To make the correct modelling and evaluation decisions, you need to think about the needs of the particular application, and make the choices that meet the requirements.
Having said which, I am very much in favour of probabilistic models and proper scoring rules, it is just that they are not the (full) answer to every classification problem (and neither are SVMs or DNN).