I've had similar callouses on two fingers. I filed them down over a period of a couple of weeks and have had no problems since then.
I've also come to the conclusion that washing pots a couple of times a day does wonders for keeping fingertip callouses in a great condition.
I wish this forum had a "block user" feature. Possibly I'm not the only one......