It's almost certain that the system isn't receiving the charge in the proper way (i.e faulty wiring). Try contacting Nintendo's customer service online or by phone, both of which can be found on their website. Good luck.
It's my job to help the next generation, and set a good example for them. And for that I'll gladly lay down my life.
I have a feeling it's the battery. When the charging inidicator goes off, you're really supposed to unplug the system as soon as possible to conserve battery life. For the XL, charge time is initially about 3 hours, and usually dips to 2 hours when the battery gets low.
You should check how long your system lasts after a charge to get an idea of whether or not you want to get a battery replacement. Also make sure the adapter's wire isn't frayed or anything; if so, you'll need a new one.