Nexus 4, heat, problem or not?

I bet there is many Nexus 4 users here and I have a question for you!

Im really close to buying an Nexus 4 but i'v been hearing all these rumors that the device is getting warm, very warm. Also things about the clocking speeds of the cpu decreasing when its heating up. Well how many of you guys are having these issues? And do you feel that its affecting the device as a whole?