Giving smokers free electronic cigarettes may not help them kick the habit, according to new research. What helps is giving them money instead.

For a study published today in The New England Journal of Medicine, researchers gave over 6,000 smokers different tools to quit smoking, like free nicotine patches, motivational texts, free e-cigarettes, and free medication or e-cigs plus a total of $600. Overall, only 1.3 percent of smokers had quit after six months. Those who received money fared much better, while e-cigs and medication alone didn’t really help people stay smoke-free.

Although the number of Americans who smoke cigarettes is going down, smoking is still the number one avoidable killer in the US. About half a million Americans die each year because of the habit. So understanding what can help smokers quit is important and companies are interested as well, as employing smokers is more expensive than non-smokers. Companies often offer smoking cessation programs, but there’s limited research on what techniques work, especially with smokers who aren’t motivated to quit. Today’s study shows what works and what doesn’t with a large number of smokers, but some researchers caution about drawing firm conclusions on whether electronic cigarettes help or not. More research on that is needed.

“It suggests that e-cigarettes aren’t a magic bullet that’s going to automatically help everyone quit anymore than other medicines are, but it doesn’t mean that they can’t be helpful for some people,” says Nancy Rigotti, a professor of medicine at Harvard and the director of the Tobacco Research and Treatment Unit at Massachusetts General Hospital, who was not involved in the study. “We don’t know that for sure, but we definitely need to find that out.”

As more and more people vape, public health officials all over the world have been debating over the health effects of e-cigarettes and on whether they should be used to help people quit. While in the UK, cigarette smokers are encouraged to switch to e-cigs to kick the habit, the US Centers for Disease Control and Prevention says that there’s not enough evidence to make that recommendation. Study results vary: research published last year in BMJ showed that vaping may help smokers quit, but another study done by the CDC showed that most adults who vape to quit also keep smoking regular cigarettes.

Today’s study adds evidence to the debate, and it’s not in favor of e-cigs. “Proponents of e-cigarettes, including but not limited to the companies that make them, argue that e-cigarettes help people stop smoking,” says lead author Scott Halpern, associate professor of medicine at University of Pennsylvania’s Perelman School of Medicine, in an email to The Verge. “This study provides by far the most compelling evidence about this critical question, and suggests that offering people e-cigarettes for free doesn’t help people quit.”

For the study, Halpern and his team automatically enrolled 6,131 smokers working at 54 companies in the US; the employees who didn’t want to take part in the study had to actively opt out and 125 did so. The remaining 6,006 were randomly assigned to different groups: some only received information on why it’s good to quit, coupled with motivational texts; others were also given free medication like nicotine patches and gums, or free NJOY e-cigarettes; others yet got free medication or e-cigs, plus a total of $600. One group, however, had the money in their account removed if they didn’t meet certain milestones.

“The rewards were a pure ‘carrot,’ and the redeemable funds were designed to be interpreted as a bit of a ‘stick,’” Halpern says. “We thought this might work better because people are typically more motivated not to lose something they have than they are to gain something they don’t have.”

After one, three, and six months, participants sent the researchers either blood or urine samples to show that they had quit smoking. Here’s what the researchers found: overall, only 1.3 percent of smokers had kicked the habit after six months, but the results varied. Quit rates doubled and tripled for smokers who received the $600, while “neither free cessation aids nor e-cigarettes did anything at all,” Halpern says.

Those percentages are very low because the people enrolled in the study were not cherrypicked based on whether they actually wanted to quit. As expected, quit rates were higher for the 1,191 smokers among the 6,006 who “actively participated in the trial,” according to the study. These are smokers who actually logged into the study’s online portal, and learned what their goals were to redeem the money, for instance. These subjects were four to six times more likely to kick the habit than the others, and that’s not surprising. These smokers were more motivated to quit to begin with. Among this subset, almost 5 percent who used e-cigs quit, compared to 3 percent of smokers who used nicotine replacement medication. That’s a tiny difference and not statistically significant, so the results “need to be interpreted cautiously,” says Anthony Alberg, an expert in tobacco control at the University of South Carolina’s Arnold School of Public Health, who was not part of the study.

“It gives a hint that e-cigarettes could be more effective, but we don’t know how many who participated actually even used the nicotine replacement therapy or e-cigarette so it makes it really hard to know,” says Alberg.

The findings aren’t the final word on whether vaping can help smokers quit, Rigotti says. Ana Navas-Acien, a physician and epidemiologist at Columbia University’s Mailman School of Public Health, agrees. Maybe the NJOY e-cigs used in this study weren’t very effective, but there are lots of different electronic cigarettes on the market and some, like Smok, might be better because they deliver more nicotine, which might help smokers stay away from regular cigarettes, Navas-Acien says. (The researchers used NJOY because when the study began, these e-cigs “were among the most commonly used products on the market,” Halpern says. The company also provided the e-cigs for free, though it had no role in the study design or analysis, the authors write.) The participants also didn’t receive any counseling on, for instance, how to use the electronic cigarettes, which might affect how much nicotine they got while vaping, Alberg says.

“Caution in this type of research is always very important,” Navas-Acien tells The Verge. “I would not claim that this study confirms that e-cigarettes are not successful to help people quit.”

For Alberg, the biggest takeaway is that the study reinforced just how hard it is to kick the habit. “There’s a desire to quit: adults know that it’s not a good thing, they know there are serious health consequences,” he says. “But it’s just such a powerful addiction.”