Benchmarks show Samsung's Exynos 5 processor is a beast with Linux
Keep in mind this is with Ubuntu 12.04 not being very optimized for Cortex A15 (it only has some support for ARM features). It should work even better with Ubuntu 13.04, which will get full support for A15 (in spring). Ubuntu 13.04 is also supposed to be focused on battery efficiency.
A quad core A15, or a big.Little 4.4 chip should run pretty nicely with Ubuntu. We may see a bigger merger between Android and Linux in the future, as smartphones and tablets become more powerful and you can dock them.