Skip to main content

Verizon LTE torture test: Why 4G can't replace your DSL (yet)

Verizon-LTE-feature-1024
Verizon-LTE-feature-1024

By Sean Hollister, with Chris Ziegler and Joanna Stern

I have a love / hate relationship with Verizon LTE.

Our first tryst occurred in Las Vegas, as such things do, over ten months ago. I was a newly seasoned Engadget editor attending my first Consumer Electronics Show with the publication; "she" was an LG VL600 USB modem. While every other editor surfed the waves of 802.11 Wi-Fi or jacked into a physical Ethernet port, I opted to try the LG — a temporary present from a MacBook-wielding editor who discovered that OS X drivers weren't yet available. Luckily, I had a Windows machine.

I plugged in the dongle, and installed the drivers from CD. I launched Chrome; My homepage loaded instantly. One 33 megabit per second (Mbps) Speedtest.net session later, I was hooked on LTE.

Seduced by a killer connection

We had some wild times together in Vegas, doing things I'd never conceived of doing on a cellular data connection before. With nearly twice the bandwidth of my wired home internet connection and latency like I'd never seen without a physical cable, I could effortlessly stream multiple 1080p videos, play OnLive games, and even remotely connect to my desktop home computer and control it fairly well.

Originally, I figured the honeymoon wouldn't last. "What happens in Vegas stays in Vegas," they say — and Verizon's superlative CES blanket of LTE surely did — but more importantly, Verizon never really promised such speeds to begin with. As new data networks pile on more subscribers, the speeds tend to dip, and Verizon had the foresight to set relatively low expectations to begin with. Even though Verizon whitepapers list 100Mbps peak theoretical download speeds (and a similarly fantastic 50Mbps up), the carrier quotes 5-12Mbps downlinks and 2-5Mbps uplinks in all its marketing materials as what you can expect in the real world.

Verizon-lte-sim-300
Verizon-lte-dongle-300

That may not sound like a lot, but let me put things in context for a second. According to content delivery network Akamai's latest "State of the Internet" report, the average US broadband connection is just 5.1 megabits per second. That's enough to play back Netflix and YouTube 1080p content, which tops out at around 5Mbps. Assuming low latency, such a connection will handle gaming on a cloud server, too — both OnLive and Gaikai require about 5Mbps to stream a playable 720p title into your home. Even crisp 1080p iTunes movie trailers (around 10Mbps each) are within the realm of possibility here... and I haven't even begun to describe what you can do with the uploads.

Today, it's certainly possible to get decent landline upload speeds in the US if you live in certain areas, but connections are usually asymetrical to a fault. What do I mean? The expensive Comcast Extreme 105Mbps cable service only provides 10Mbps uploads, and my own AT&T U-Verse 18Mbps fiber-to-the-neighborhood solution only does 1.5Mbps up. Meanwhile, my cellular LTE dongle managed speeds as high as 17 megabits per second out in the field, making it invaluable for quickly uploading HD video from my DSLR camera.

With all that LTE offers, why do I hate her so? When I took her home, she turned out to be as fickle a connection as any I've ever known.

LTE explained

Verizon-lte-explained-555

But let's step back for a second. What is LTE, and where did it come from? For something as complicated and contentious as a new wireless standard, the journey from draft specification to commercial availability is an extraordinarily long one, usually taking half a decade or longer. Indeed, LTE was no different — Verizon's deployment represents years of international collaboration. Waiting for new, high-speed networks is never fun, but the end result is a standard that a lot of people agree on: carriers on pretty much every continent are either working on LTE networks or have them on the roadmap, which ultimately leads to reduced component costs and better handset availability for everyone.

Interestingly, the differences between LTE and its chief competitor — WiMAX — are relatively small. Both technologies employ something called OFDMA on their "downlinks," the connections that send data from the internet down to the phone; that's the connection that really matters, since we're spending far more time downloading data to our devices than we are uploading. OFDMA stands for Orthogonal Frequency Division Multiple Access, and it's a scheme for allowing many users to share the same bandwidth at once, just as CDMA (which is used by 3G services like EV-DO and UMTS / HSPA) and TDMA (used by GSM) are. Without getting into the nitty-gritty details, OFDMA is more spectrally efficient than either of those other schemes, which means you can fit more data (and more users) into the same amount of bandwidth. These days — especially as the FCC and other spectrum agencies around the world are struggling to free up airwaves for broadband use — that's exceptionally important.

Interestingly, the differences between LTE and its chief competitor WiMAX are relatively small
Lte-logo-300
Verizon uses FDD-LTE on 700MHz spectrum

Sharing bandwidth isn't just about dealing with multiple users at once, though. There's also the issue of splitting the airwaves between the uplink and downlink, and that's another matter altogether. Combined, these two types of multiplexing comprise something called the "air interface," which is one of the fundamental attributes that defines a particular wireless technology. 2G GSM, for instance, is a TDMA FDD (frequency-division duplex) system, meaning it employs TDMA on two distinct regions of bandwidth to manage the uplink and downlink independently. Like most wireless standards, LTE began as an FDD standard, but some countries and carriers ran into a problem: it's really difficult to free up one range of spectrum to deploy a wireless service, much less two ranges, which is what FDD requires. That led to the development of TD-LTE, which instead uses time-division duplexing between the uplink and downlink —- in other words, the phone and cell tower rapidly switch directions while "talking" to each other so they can send data back and forth on the same channel and the same range of spectrum. It requires less bandwidth to deploy than traditional FDD LTE does, and China — along with Clearwire in the US — have led the charge in its development.

That said, Verizon's LTE service uses FDD (as does AT&T's), and it's further along in the development process than TD-LTE, which means better access to devices sooner. One need look no further than the carrier's extensive lineup of LTE-capable devices for evidence of that. Both Verizon and AT&T also use the recently-auctioned 700MHz spectrum for their deployments, though with one key difference: AT&T uses a mixture of 700 and 1700MHz, while Verizon is purely 700MHz at this point. Why is that important? Lower frequencies are able to travel further with less signal loss, which means that a 700MHz network can be deployed across a wider area with fewer towers (and, consequently, with lower expense). Verizon has committed to a 100 percent overlay of its existing EV-DO network in 2013, and it stands to reason that the 700MHz spectrum's characteristics play a role in that aggressive schedule. What's more, lower frequencies have better building penetrability, so you're more likely to hold a signal deep inside a structure with 700MHz than you are at higher frequencies; that's been a problem Sprint has been dealing with over the past couple years with its WiMAX network, which is deployed all the way up at 2500MHz.

LTE in reality

Nine devices, three cities, one giant test

If a tiny stick or a portable hotspot could give you high speed internet wherever you go, why would you bother paying for a landline at home? That's the thought that occurred to me, and so I decided to cut the cord.

I unplugged my Ethernet, turned off my Wi-Fi and got my hands on every single Verizon LTE device sold, including the Pantech UML290, Novatel USB551L, Samsung SCH-LC11 Hotspot, Novatel MiFi 4510L, LG Revolution, Samsung Droid Charge, HTC Thunderbolt, Motorola Droid Bionic and even the Samsung Galaxy Tab 10.1 LTE tablet. [The Motorola Droid RAZR and HTC Rezound weren't on sale when we began our test suite, but we'll be adding them over the coming weeks!] Then, I realized that testing from my residence in California wouldn't mean much on its own, so I conned fellow editors Joanna Stern and Chris Ziegler into doing the same from their workplaces in New York and Chicago.

We lived with LTE as our primary internet connection

What follows are our observations after living with LTE as our primary internet connection, testing each and every device across three different urban LTE markets, with a minimum of 45 speed tests per device per person: 15 in the morning, 15 in the afternoon and 15 in the evening, not counting all our non-quantifiable observations. We measured upload and download speeds, latency and reported cellular reception during each individual test for a total of over 5,000 data points, suffered spontaneous connection failures, enjoyed amazing speeds and lived to tell the tale. In other words, we're not scientifically proving anything here, but we've spent a lot of time with Verizon LTE, and it can be both an amazing and frustrating experience. Oh, and we're not done yet. We'll be testing future milestone LTE devices as we get them.

Lte_stack_300
Verizon-lte-dongles-300
Verizon-lte-mifi-300

Speed

Verizon-lte-results-800
Let's get this out of the way: whether you live in Chicago, Manhattan, or San Jose, LTE speeds are pretty great

Average Download [mbps] Average Upload [mbps] Latency [ms]

10.51 5.83 60
San Jose 9.51 5.68 50
New York 10.51 4.49 70
Chicago 12.32 7.63 67

Let's get this out of the way: whether you live in Chicago, Manhattan, or San Jose, LTE speeds are pretty great. We averaged 10.51Mbps down and 5.83Mbps up across our three test sites, and latency of 60 milliseconds. Some cities were more average than others, though — our San Jose office managed just 9.51Mbps down and 5.68Mbps up on average, while New York devices managed 10.51Mbps and 4.49Mbps, and Chicago took the lead with 12.32Mbps and 7.63Mbps speeds. However, part of that may have to do with building construction — when I moved my devices from my San Jose home office to my living room, I could often increase their speeds significantly. That extra building penetration of the 700MHz frequency seems to have its limits. Either way, San Jose actually pulled ahead when it came to ping, with average latencies of just 50 milliseconds, compared to 67ms in Chicago and 70ms in NYC.

Still, if your LTE is slower than the Joneses, hardware is probably the culprit. We found we could pull out two different LTE devices at the exact same time and place and get totally different results. Take a look at this chart:

Average Download [mbps] Average Upload [mbps] Latency [ms]
HTC Thunderbolt 8.25 4.12 76
LG Revolution 12.89 6.76 59
Samsung Droid Charge 9.93 5.65 65
Motorola Droid Bionic 6.21 5.00 61
Galaxy Tab 10.1 LTE 13.62 7.65 61
Samsung SCH-LC11 13.81 7.80 58
Novatel MiFi 4510L 10.67 7.09 55
Pantech UML290 7.00 5.30 56
Novatel USB551L 12.53 3.56 54

As you can see, average speeds are a little different from device to device. Even so, this chart doesn't tell the whole tale, because speeds also vary quite a bit even when using the exact same LTE handset or dongle over and over again. We're going to do something a bit unorthodox, and use a candlestick chart (typically used for stock prices) to show you how speeds differed over the 135+ tests we performed on each unit. The thin line depicts the full range of download speeds we got, from maximum to minimum, while the solid blue bar is one standard deviation above and below the median. That might be Greek to your ears, but it's actually fairly easy to read: longer bars mean less consistent performance, and the lower the thick bar goes, the slower its typical speeds.

Lte-candlestick-800

You'll note that almost all of our devices have fairly long, thick bars, and almost every single device had terrible speeds at one point or another. There's not necessarily anything to worry about, but it just goes to show that LTE speeds aren't all that stable yet. Remember, we're still working with cellular radios here.

Looking for the Motorola Droid RAZR or HTC Rezound? By the time they arrived on our doorstep, we were nearly done testing, but we've reviewed both and have some initial results for you. The RAZR gave us anywhere between 5 and 15Mbps down (and 5Mbps - 8Mbps up) in New York, and the Rezound managed to maintain over 10Mbps downloads in our Chicago office. We'll be running the full test suite on both soon and will add them to the charts accordingly.

Reception

During each and every speedtest, we dutifully copied down how many bars of reception our devices saw. Unfortunately, we quickly discovered that the bars weren't particularly useful for predicting how well LTE would perform. In the exact same location, LTE devices with fewer bars could out-perform others with more, and we had quite a few unusably slow or completely unresponsive data connections even when our hotspot had full bars.

Lte-reception-chart

Interestingly, the definition of "full bars" varies from one device to another — the LG Revolution and Droid Charge, for instance, have up to five bars of reception, while the rest have four. Still, even if the bars aren't telling the whole story, signal strength can make a difference. Moving from a small closed room to a wide open one, we saw uploads in particular improve drastically. Not that it necessarily means anything, the Novatel MiFi 4510L had the best reception in our tests, with three out of four bars even in that small closed room, while the Samsung SC-11 and both USB dongles seemingly tailed the pack, sometimes displaying just two or three out of four bars in places where other devices were maxed out.

Reliability

The fastest devices seemed to be the least reliable in our tests
Lg-hotspot-300

Speed is all well and good, but what does it matter when you're getting booted off the web? Interestingly enough, we found the fastest devices were often the least reliable in our tests. The LG Revolution and Samsung Galaxy Tab 10.1 often pulled down over 20Mbps, and the Samsung MiFi gave us our all-time best download speed of over 40Mbps on a fluke, but these devices frustrated us continuously. Our LG Revolution and Samsung MiFi seemed to have a habit of spontaneously dropping the internet connection every so often, even though the device would sometimes still display the 4G logo and full connectivity as if nothing were wrong. Meanwhile, the Samsung Galaxy Tab, Droid Charge and HTC Thunderbolt could get into a state where they endlessly looped between "connected," "disconnected," and "connecting" with no internet access whatsoever in the meanwhile.

At least, that's how they performed in San Jose. My colleagues in Chicago and Manhattan had some of the exact same issues, but typically far less often. We're not sure why, but you might be able to chalk it up to the Silicon Valley's sprawling terrain and perhaps a more saturated LTE network.

We've also seen several devices get stuck in 3G mode for anywhere up to an hour, after dropping the LTE signal for only a moment — although that may be preferable to the way the 3G/4G handoff worked before. When I started testing LTE months ago, a device at the edge of a coverage zone would switch back and forth uncontrollably between good CDMA and bad LTE in an attempt to hop onto the faster network... but like the dog and its reflection, it was usually at the expense of both.

The dongles didn't fare all that much better, and there seem to be some serious issues with the Mac drivers and / or VZAccess Manager desktop software here. All our Windows and Mac PCs had intermittent difficulty recognizing the Pantech and Novatel when they were first plugged in, and sometimes we'd suddenly find ourselves disconnected from the network in the middle of a session with the infuriating message "Connected - Dormant" or "No wireless device detected." These usually required us to yank the dongle and restart the software to fix, which proved particularly problematic on a OS X computer with the Novatel USB551L, as every time we removed the dongle, our MacBooks spontaneously crashed. No joke. Perhaps we got a bad device, but it wasn't a fluke; we tried it again and again across several machines, reinstalled drivers and software, and got the same result. Most of our USB551L results were gathered on Windows 7 accordingly. Out of all our devices, the Novatel MiFi and Droid Bionic did best, though even they weren't without their quirks. My Droid Bionic got stuck on 3G for an hour, and my Novatel 4510L dropped the connection twice in a day's worth of regular use.

Under duress

So far, we've told you what it's like to use LTE devices in tightly controlled indoor environments — offices, apartments and houses within solid coverage zones. What is it like to venture beyond? We've got some anecdotal evidence, both good and bad.

Outdoors

The wireless industry likes to talk up Verizon LTE's 700MHz frequency because it's supposed to penetrate buildings better than the competition, and while that may be true, I did notice a boost in speed and reliability simply by venturing outdoors. I also took a couple of drives and a train ride or three, and discovered that Verizon hands off the LTE connection from tower to tower fairly gracefully these days. I managed to make it from San Francisco to San Jose with the Novatel 4510L MiFi and Droid Bionic with only brief interruptions in internet service and some time spent on 3G, though both devices drained their batteries more rapidly.

Trade events

Back in April, when LTE was relatively unused, we could set up a hotspot and upload pictures and video directly from an event, as speedily as a home connection. In more recent months, we've seen LTE fail time and again. At the Electronic Entertainment Expo in June, I found LTE reliability spotty in general and next to non-existent during keynotes, while Sprint 3G did far better. So did Verizon 3G, after I switched one of the devices I was carrying away from the LTE network. At Microsoft's Build conference in September, I couldn't load a webpage on any of the four LTE devices I was carrying, and found myself at the mercy of local Wi-Fi, which thankfully turned out to be excellent.

What is it like to take Verizon LTE outside the office walls?

Battery life

It's not exactly a secret that the current generation of LTE devices suck down more battery juice than most, and if you're planning to purchase one you'll probably want to know which device can squeeze out the most oomph. It's particularly annoying because of how difficult they can be to charge, but more on that in a moment. While we didn't manage to test the exact impact of USB dongles on our laptops — there's a noticeable dip in battery life, to be sure — we did run down the battery of every single phone and Wi-Fi hotspot with our standard battery drain test, which endlessly cycles through a series of high-res online images and popular websites. Here are the results:

Battery Life
HTC Thunderbolt 4:19
LG Revolution 3:53
Samsung Droid Charge 3:56
Motorola Droid Bionic 4:21
Samsung SCH-LC11 3:18
Novatel MiFi 4510L 3:21

As you can see, it's fairly cut and dried with Verizon's first six battery-powered LTE devices. You can expect three hours of casual browsing with the tiny battery in a hotspot, and four hours with a smartphone. We also gave the Galaxy Tab 10.1 a try, and it looks like you can get over ten hours with the relatively hefty lithium ion pack that a tablet has inside, assuming that the hefty screen stays off.

Quirks and tips

Verizon-lte-mifi-charging-555

Remember last paragraph, when I said LTE devices could be difficult to charge? That's because you won't necessarily be able to do so from your standard USB port. While running both Wi-Fi and LTE radios at the same time, we noticed some of the smartphones drew more power than our laptop USB sockets could put out, slowly draining the battery rather than charging it up. Even if you've got a wall socket you might need a tablet-class charger that provides well over one amp to keep it from dying during use. Turn off the Wi-Fi hotspot and things get better, but don't expect to easily be able to tether an LTE device to your PC. There aren't any native drivers for Mac — though apps can help — and neither the Samsung nor Novatel hotspots allow USB tethering at all. Oh, and the Novatel actually won't charge when plugged into a computer and providing LTE. You'll need to plug in a wall wart or find a micro-USB cable without data leads to simultaneously use and charge.

Mind you, with all that electricity coursing through those tiny cases, these devices also have a tendency to get rather hot. Not quite hot enough to burn, but enough that it can adversely affect performance. For example, after a long day of testing with the Pantech UML290, we noticed that we were only pulling down 1Mbps. Clearly, something was wrong, but after unplugging the uncomfortably warm dongle and letting it cool for a few minutes, we were getting 8Mbps again. We had similar episodes with other devices, including the Samsung MiFi and Droid Bionic.

Though the situation seems much better than several months ago, it seems there are still edge cases where Verizon LTE devices won't negotiate the 3G / 4G handoff well, leaving you stuck on 3G in an LTE zone or vice versa. What to do? There are a few tricks that you could try, depending on what you're using. If you've got a supported smartphone, give the LTE OnOff app a go — with a couple presses, you can force your device into 3G or LTE mode and keep it there as long as you like. USB dongle owners can find something similar in a hidden VZWAccess Manager debug menu; just hit Ctrl-D and type in the password "diagvzw" for access. Meanwhile, the Novatel and Samsung hotspots have web configuration menus — type 192.168.1.1 in your web browser URL bar when connected, and enter in the password "admin" for the Novatel or the wireless password that came with your Samsung (it should be a string of digits). Unfortunately, the Samsung won't let you choose between LTE and EV-DO, but you might want to tweak a few other variables while you're in there. By default, both hotspots have a 30 minute timeout before they turn off, and some users report fewer problems when disabling 802.11n Wi-Fi and using 802.11b/g instead. Neither of these solved our sporadic issues, though.

Protip: switch to 3G when LTE's not working
Verizon-lte-dongle-300-2

Wrap-up

Verizon-lte-main-800-strip

Verizon has "the nation's most reliable wireless network." You've heard it time and again. The company's had over seven years to build out an EV-DO network, and it's hard to expect Verizon to perfect LTE at the drop of a hat. You could also argue that we're expecting too much out of LTE. Verizon's pricey service plans currently max out at 20GB per month, a pittance compared to the data buffet many expect from their home networks. I don't know about you, but I'm currently brushing up against my 250GB AT&T U-Verse cap right now. Besides, LTE's not designed for high-speed access 24 hours a day, seven days a week anyhow, right? Tell that to Verizon, which is planning to launch fixed residential LTE connections by the end of this year to supplement its FiOS fiber optic network. It's only a matter of time before we expect wireless connections to replace physical cables, and Verizon isn't quite there yet.

But blaming our failed experiment on Verizon's airwaves (or our unfettered expectations) wouldn't be completely fair, because it seems likely that hardware and software issues with these devices were also to blame for some of the failings here. Qualcomm, LG, Samsung and Motorola all have their own LTE radios in these devices, and save Motorola, we've now tested at least two of each. Each demonstrated its own particular quirks, many of which inexplicably kept us from enjoying LTE's blazing fast speeds. Even if Verizon's network was magically made perfect tomorrow, these devices might hold you back.

We're also not sure why such an advanced technology has such an iffy software client for its USB modems, or why hotspots can't tether to save battery (unlike many of their 3G counterparts) or, for that matter, why Verizon's forums are filled with complaints marked "solved" when they're not. By pushing LTE so far so fast without addressing issues head on, Verizon's counting on consumers to beta test its service and possibly leaving an opening for rival networks to woo customers not immediately taken with the new network. Verizon's way ahead of the competition with LTE, and we're hoping reliability will follow — we don't have reason to doubt that it will — but AT&T's gunning for Verizon now, and Big Red's lead isn't bulletproof. We saw very passable speeds on the HTC Vivid, AT&T's first LTE smartphone, and even HSPA+ and WiMAX are delivering respectable rates on the download these days, as we noticed in our reviews of the Amaze 4G for T-Mobile and the Photon 4G for Sprint.

Verizon LTE isn't what we'd hoped it'd be right out of the gate, but the future is bright indeed. Cheaper, cooler, faster next-gen chipsets are due early next year from vendors like Qualcomm, and later in 2012 Verizon can start phasing out some of the legacy network (and thus issues switching between networks) as it migrates to Voice over LTE. Then starting in 2013, carriers are looking to start rolling out LTE-Advanced, promising theoretical transfer rates of up to one gigabit per second. Cellular networks are undergoing a major transition right now, but unless the world adopts fiber more rapidly, cellular prices stay sky high, or LTE issues are deeper seated than we expect, there will indeed come a time when a wireless network could replace your home internet connection.