Mp3 stands for MPEG-1 Audio Layer 3 m and p coming from mpeg, and the 3 from being the 3rd layer of audio compression, it's popular because it is highly compressed, but replicates uncompressed sound quality.
Mp4 is actually mpeg4 video. Mp just a short version without the eg. Compressed video, usually at small sizes and resolutions for ipods, psp's and etc.