The only solution not involving code is:
Create the column as:
- Choice
- Not required
- Unique
- Choices: 1, 2, 3, 4 and 5 (not None)
- Drop-Down Menu
- Clear default value
![enter image description here][1]
That'll allow the users to create any number of entries without a priority, but only one for each of the 5 values.
This seems is like you want it to work. BUT once an entry has been assigned a priority, there is no way to clear it again. [1]: https://i.sstatic.net/izqts.png