Capital One, No Hassle Rewards Charged me 68 dollars on a card that hasn't been used or activated!! Detroit Michigan

I applied for CapitalOne credit card in Dec 2007. Before applying I called and specifically asked if there was an annual fee associated with the card offered (although the offer letter said there wasn't any I just wanted to make sure). I was told that there was no annual fee, so I went ahead and applied.
My good luck I forgot to activate this card.

For two months I haven't gotten a bill from them, only bogus invitation to join this'n'that. After that Poof, the slap me with a 39 dollar anual fee and a late payment of 29.

I called customer service that same day, after being bounced for an hour I was finally able to speak with a supervisor. She assured me that my account would be closed after, and I quote here "the end of billing cycle + 1 month" (is that even legal?). She has agreed to take off that annual fee and the late charge from my account, instead I had to pay 15 dollars which she assured me would be rebated to me after account was closed. She also told me she made sure there would be no further fees and bogus charges on the account.

Having read so much about them now I am still worried that they will slap me with another charge or come up with another way to rip me off.

And all this HASSLE for a card that I DIDN'T USE OR ACTIVATE!!! NEVER EVER APPLY FOR A CARD!!!!!!!!

Dia
Detroit, Michigan
U.S.A.