Hmmm. I begged and pleaded with Ben to get rid of my Interrogation or Intimidation, I forget which. He agreed to do it when I said I wasn't asking for the skill points back, just to get rid of the skill, which I thought was nice of him. Then I couldn't have that stupid accident happen.
But LATER, I rebought the skill when I was experienced enough that it made sense to have all the skills. Now I have trained myself to judge the character before I use the tough or charming skill.
So maybe eventually you'll be glad you have it?