How do I really delete images and why aren't they really

How do I really delete images and why aren't they really deleted? I upload an image "image.jpg." Then, I delete the image. I create a revised "image.jpg" same filename and upload it. And, the original image still appears. And, this is not a browser cache as I can clear the cache or change browsers and I get the same result. TIA