Is it too late to start karate?

I used to do karate when I was younger (9 years old) and had reached green belt. I am now 15 and i was thinking of rejoining. Would I be at too much of a disadvantage if I started again now, even though I would progress through the first few belts quickly w/ past exp., or are there beginners of all ages. All advice is welcome, 10 points for most helpful answer.