Mine does it too, so I supose it's normal. It's bad practice though. The case and cover are both made out of plastic, so they've basically insulated the PCB. Even if it's only disipating a small amount of power, it is going to get quite toasty.
I tried to take the cover off of mine, but it's glued at the far end, so I could only peek inside. There is a surface mount power component in what looks like an SOT-223 package. Couldn't get the number off of it, but I bet that's where the heat is coming from. Typical max die temperatures on those devices are 100 to 150 degrees C. (212 to 305 F)
The good news is if the bottle neck is in the plastic case, then the temperature rise from case to die is probably not that large. Especially if they used multiple or thicker ground planes in the PCB and grounded copper 'pours' into the unused spaces on the top & bottom layers to act as heat spreaders. That costs little or nothing to do. It's just a matter of thinking about it while laying out the PCB. It won't make the outside case any cooler but it will make the chips run cooler, which is what matters.
Remember the temp sensor in human hands is very non-linear. (I think we evolved them so we can tell when someone is running a fever) Silicon ICs and steel tanks are perfectly happy operating at much higher temperatures than humans can tolerate.
As long as you don't smell smoke or decomposing plastic at the USB connector, it's more of an image problem than a technical one.