My only tip where this is concerned is don't use hot glue. I know there are a lot of people here that do, and love it, but a lot of them also run into the problems you are now. Also, when it comes to the smoothing stage, if you sand down too far and get to the glue, it looks like crap then too (if you watch Cereal Killl3r's videos and build thread, you see him cursing the hot glue as he's sanding his helmet and torso as well).
I use Elmer's white glue. It's cheap, water-based so it's non-toxic, won't burn you or leave behind a ton of strings and residue that looks terrible. If you're not from the US, then look into pva (white) glue.