The effect of 64-bit on Android?

Next year, we would see the release of the 64-bit ARMv8 processor from Qualcomm, Samsung and Nvidia. Of course, with it comes the 64-bit-equipped phones. Do you think this would affect Android? Would it become more fragmented? How can it affect older phones? Would it become slower on 32-bit phones?

On a related note, I'm seriously tempted with the Nexus 5 (Assuming the rumored specs are true) but I'm worried about its performance in the future because I'll be using it for about 4-5 years. Is it worth it? Or am I better off with next year's Nexus phone?