I've always been told it was a product of having the gears slightly misaligned. This can happen if the bearings were pressed on too far, or not far enough on the gear shafts, or if the gears themselves were pressed on to incorrect tolerances. I guess it might also be possible that a significant jolt to the transmission could move the gear on the shaft slightly.
At some point my 2nd gear started to whine. I'm not sure if it coincided with a change to the transmission I did, or me removing a whole bunch of stuff that might have masked the sound in the past. Either way it is very minor in my case and only at low speeds (the sound goes away above 2.5k rpm) so I haven't even begun to try and think about fixing it or even worrying about it.