MP3 inserts silence as part of the encoding process.
http://lame.sourceforge.net/tech-FAQ.txt
"1. Why is a decoded MP3 longer than the original .wav file?
Because LAME (and all other MDCT based encoders) add padding to the
beginning and end of each song. For an explination of why, see the
questions below.LAME embeds the amount of padding in the ancillary data of the first
frame of the MP3 file. (LAME INFO tag). The LAME decoder will use
this information to remove the leading padding of an MP3 file. "
You can get tricky and try to calculate how much padding there is at the start, but then you risk chopping the first few samples off as the decode process will decode from silence to the first sample data’s values, so you dont also want an abrupt start.
MP3 is not a good archival format at all. You should use flac/ogg or wav to avoid this problem.