Take a small micro screwdriver or even a straightened out paperclip and you will see a hole on the windshield part of the mirror mount. You will need to push the tool or clip in that hole so that it compresses the spring mount holding it there. It should then slide up and out as you push this. Try it and let me know.
Incidentally, where did you get a replacement mirror? I need one too!