Okay, lets get some termanology here, the first picture is a spindle with the bearing assembly in it, the second picture is the hub with the bearing race(yellow arrow) stuck to it. You said something was 'too big', so it must be the hub that is too small on the end where it has worn(the end where it touches the cv joint), or the bearing must be too big and improperly sized. Just to clarify things, this picture is before you put the spindle back on the car correct?
The balls will probably never touch the spindle, they may however touch the hub and wear the shaft down if the bearing race ever breaks. Shit gets hot, especially when the correct clearance isn't maintained(with correct torque, or momentum as you put it).