Switching from PC to Mac - how to deal with Ext. HDs?

After having partially switched from PC to Mac (with the purchase of a Macbook Air last summer), I've now gone all in after getting a GREAT deal on a 2011 iMac.

The problem is...I have multiple external hard drives that store all of my photographs (I'm a prolific hobbyist photographer) and music, videos, etc. The problem is, these drives are all formatted in NTFS (which is not writeable by a Mac). I need to convert them to something the Mac can read/write to.

Ideally, I wouldn't have to wipe the drives to convert the file system.

FAT32 is the obvious choice, being supported by Mac and PC (with the added benefit of not having to wipe the drive to convert it). However, it has limitations (4GB max file size, not FULLY supported by Mac, etc).

What should I do here? I rarely have files that are above 4 gb (but could in the future if I get into video). Are there any performance (speed) downsides with going to FAT32?