Tired of my First Premier Bank Visa Credit Card

I am holder of the First Premier Bank Visa credit card. You know, the one with the $250 credit limit. I'm very unhappy with this card. I've had the same low credit limit for almost two years. I'm tired of paying $7 to make a payment online. I'm tired of paying the annual fee monthly instead of annually. As of now, I know I have a balance on the card, can you please tell me the steps to closing my account without hurting my fragile credit score?