How can I fix a permalink error?

A permalink issue has occurred a couple of times for my sites. The cause is probably that I have previously had a page with the exact same permalink. Example:

http://beardface.net/safety-razors
becomes
http://beardface.net/safety-razors-2/

There are no other page with this permalink and all caches has been cleared with W3 Total Cache.

It is not possible to simply edit the permalink while editing the pages. The update results in the second permalink again. If I select the first permalink it will, however, redirect to the second.