I knocked out a mirror and bought an unpainted aftermarket OEM-replica. Is it possible to swap the painted part of my damaged OEM mirror with the same part on the aftermarket one, or would I have to get it painted?

Would it look crappy if I spray painted it? Are there any high quality spraypaints that you guys can recommend that will last a long time and won't fade or crack?

You'll never find a spray paint that will match the factor color or quality. Even if you took it to a professional body shop, they would need to blend the paint to get it so that it doesn't stand out (which would be my recommendation if you want it to look good).

if you spray paint it will chip pretty fast...so that means a lot of touch-up and it's not gonna look smooth at all.

It should be possible to swap the painted part -
I think I remember this being part of the acura mirror swap?