How fast of an internet connection do I need?

It'll mostly just be used to browse the internet, download albums now and then (legally), and stream Netflix. AT&T U-verse offers 6 Mbps, 12, 18, etc, and while I don't want to pay more than I have to, videos buffering drives me crazy. Thoughts?