Kaitlyn and I love food; take that above photo of us as proof. We’re often conflicted about leaving reviews for restaurants, however. Kaitlyn has never written a Yelp review, whereas I’ve written only one. We worry about reviews as a whole. Should everyone be a reviewer? Are people good? Do they want to intentionally ruin each others’ livelihoods?
For this week’s episode of Why’d You Push That Button, we look into restaurant reviews and why people leave them. We talk to a Yelp Elite member, Dominek, as well as a restaurant owner named Benham about how Yelp affects their lives. Then we take our questions to Brian Boshes, product manager of contributions and community, who explains why he thinks people leave reviews and whether they’re tearing apart the fabric of our society.
You can read the transcript of our interview with Boshes below and listen to the full episode above. Of course, you can find us anywhere you find podcasts, including on Apple Podcasts, Spotify, Google Play Music, and our RSS feed. Catch up on season 1, too, if you missed out the first time.
Ashley: Alright, we are back. Kaitlyn and I are here, and we have a guest. Brian Boshes, the product manager of contributions and community at Yelp. Hey, Brian.
Brian Boshes: Hello.
Ashley: Thanks for joining us.
Yeah, thanks for having me.
Ashley: This episode, we’re talking about leaving reviews, particularly of restaurants. And before we get knee-deep into Yelp and its strategies, and how the product changes and all of that, can you just explain a little bit of what you do at Yelp?
So, as the product manager for contributions and community, I have kind of a twofold job, but most of it centers around engaging Yelpers to participate on the platform, so I’m building also very special experiences for our Elite Squad, and that’s where some of the community aspects come into my job.
Kaitlyn: What are the reasons that people use Yelp?
I think the three main ones that we tend to categorize most of the feedback we get when we ask users why they contribute to Yelp. The first one is altruism, and that tends to be the biggest one so there’s this idea that they’ve gotten so much out of Yelp, and they want to give back to the community, or they’re champions for their local community and they want everyone around them to have the same great experiences that they’re having.
The second one is experience, so these are your food journalers, or your food bloggers, or people who are really participating in Yelp mostly for their own benefits. They like writing about their experiences. It’s kind of like their experience diary if you will.
The third theme is very feedback-driven. Thankfully, most of it is positive feedback, but it’s giving feedback either to other people, or mostly to the business themselves, so a thank you of what a great experience they had, or some suggestions for improvement.
Ashley: Do people tend to come to Yelp for positive reviews? Or do you find this 50/50, like if people have a negative experience, they want to tell the world about it?
Yeah, I mean thankfully around 80 percent of the reviews on Yelp are your three, four, five stars. The feedback, the reviews on Yelp tend to skew very positively — five star obviously being the biggest bucket of the three, four, and five stars when you sort of break it up. Yeah, I’m happy to say that most of the feedback is positive.
Ashley: Yelp can be incredibly influential on whether people go to places or not. I am wondering how you maintain the integrity of the community. There are bots and all that stuff. So, how have you guys addressed the integrity of the community?
Yeah, the integrity of Yelp actually has always been a really important part of Yelp, and has been there pretty much since day one. And we have, I’ll say like a multifaceted approach to making sure that that quality stays high on Yelp. One of our, I would say more human lines of defense, is that anyone can report a review that they think violates our Terms of Service. We actually get a good number of reports from, both consumers and business owners saying, “Hey, I think that this review violates your Terms of Service, can you review it?” And someone on the Yelp side reviews it and if the moderator agrees, then that review gets removed.
We also have recommendation software that looks at every single review that comes into Yelp and tries to make the determination of whether that’s content that we want to recommend, and then those recommended reviews are actually what then get displayed on the site and put into the star rating.
We also have a program called the Consumer Alert Program here at Yelp, and this is a program that, if we find that someone is going through extreme efforts to sort of manipulate their star rating, through solicitation, or attempts to purchase reviews, or doing giveaways, or other things that are against our Terms of Service. We let the consumer know we just say, “Hey, there’s something fishy about the reviews on this. They look like they’re all coming from the same business owner, or they all look influenced in some way. Just to let you know, kind of give them a heads up.” As a last resort, we may take legal action against businesses or reputation management companies. You mentioned bots that are using, sort of illicit tactics to manipulate reviews. We’ve also done that in the past.
Kaitlyn: When you’re talking about reviewing something to see if it violates the Terms of Service, what exactly does that mean? I would assume that you get contacted a lot by restaurant owners or something like, “This didn’t happen,” or “This photo is not accurate,” things of that nature. What kind of verification process does that go through?
Well, our moderators, they do their best to validate the claims by both the reviewer and the business owner. I mean. we look at a lot of different factors. Obviously, if the business owner had really compelling evidence, it might be something that they can attach to their case and then that can influence the moderator’s decision, or help amend their decision I should say. In most cases, like with photos, it might be a little bit easier. For example, a person posts a photo that might be a little bit extreme or not part of a normal experience that someone would have at a business that we, for example, the fly in the soup photo is one that we use as an example. We have the ability to not necessarily delete that photo, but it wouldn’t be a part of the business’s main photo carousel, but might still be attached to that person’s review because it did reflect and experience that that individual had at a business, but might not be something we want to promote as one of the main business photos for example.
Ashley: Yeah. Prioritization is something that we were interested in. You clearly have Yelp Elite badges, things like that. How do you prioritize what reviews show up first or what photos show up first?
For photos, it’s insanely complicated, and actually, I don’t know as much about the photo piece. For reviews, it’s a multifaceted algorithm that looks at different parts of the review. I can tell you that time is still a really big factor. Recency carries a lot of weight, we know with consumers, and so the newest reviews tend to float to the top, and older reviews tend to float to the bottom. There are some other things that also go into the weighting there. Obviously, our Yelp Elite write some of the best content and some of the most trustworthy content, we think and so we tend to showcase their contributions when we can. So those are a couple of things that might affect the sorting.
Another thing that does happen as well is, you can have friends and followers on Yelp and so we know that you probably care about those people’s reviews more than someone you don’t know. So if you have a friend, if you’ve connected with someone on Yelp we’ll actually show you their content first, which is just kind of nice. You might go to a business that you’ve never been to before and then all of a sudden you see, “Hey, I know that person. Oh, hey, they’ve been here and here’s their review,” and so Yelp knows that that’s kind of something that people like, and so we prioritize that as well.
Kaitlyn: I don’t know if you can answer this, but maybe you saw a while back there was this viral Vice piece where this guy managed to make a shed the top-rated restaurant.
Yeah, the top-rated restaurant in London.
Kaitlyn: Yeah, on TripAdvisor. That piece is very riveting in that it seemed like it was pretty easy, even though everybody who came to his shed obviously hated it because it was horrible. How would you explain how that happened on TripAdvisor. Could that happen on Yelp?
I’d like to say that it couldn’t happen on Yelp. As far as your other question, how it happened on TripAdvisor, you’d probably have to get a product manager from TripAdvisor on the show to answer that. They might not want to. I think that article made the rounds internally to kind of keep us aware of our mission. Why it’s so important to keep the integrity of the content on the site at the forefront of everyone’s minds. We have our team that runs the recommendation software, but It’s kind of everyone’s job at Yelp to make sure that we stay trustworthy and that an article like that never really comes out about Yelp. They’ve done comparative studies of different sites trying to game the system, and in most of those Yelp ends up doing fairly well, so we want to continue to see that and continue to promote that in the reviewing community.
Ashley: How do you deal with angry restaurant owners? Do you just engage the community, or what do you do?
We do have a biz outreach team whose job it is to understand business owners, kind of where that anger is coming from and then hopefully channel that back into a positive way either through product changes or at least outreach to make sure that the meme is out there that Yelp’s doing what it can and we’re not out to crush small business or anything like that.
As far as being angry, I mean it’s kind of a natural human tendency that you’re going to focus, even if you’ve got hundreds of positive reviews, those couple of negative reviews are going to keep you up at night, especially if those negative reviews you feel like are unfair, so sometimes it’s just it’s hard. It’s a hard thing. It’s a human tendency to focus on that. Obviously, if there’s some constructive criticism in there, if there’s something that the business owner can use to help fix something that’s wrong with their business, obviously then we encourage them to do that. We encourage them to engage with the reviewer.
Even last week, I wrote a review for a business. It’s still a five-star review, but in the business, I was like, “Hey, the thing you sold me was kind of expensive,” and the business owner reached out. They used the free tools where they can direct message the reviewer and they said, “Hey, just to let you know it was a little bit more expensive because we were actually selling you a professional grade item that you can’t find at like a Home Depot, and the installer should have told you that, and we’re sorry if he didn’t.” I was like, “Oh, great. That’s actually really great to hear,” and I would have never kind of gotten that piece of mind had I not written the review and then had the business owner reply to me and then I went back and sort of edited the review, based on that information to be a little bit more fair.
Again, it didn’t affect the star rating, but it allowed me to clarify my review and it’s like, “Wow, that was a really cool interaction,” that if a business owner is participating in the community they can have. That’s an example from my own personal experience with that.
Ashley: It’s funny because there’s a restaurant in my neighborhood, in their bathroom, they post every negative Yelp review they’ve ever gotten so that’s interesting.
Like it’s wallpaper?
Ashley: It’s like a wall of negative Yelp reviews, yeah.
I saw once that someone turned them into T-shirts. I think there was a pizza place that put their one-star review about how their pizza had a lot of grease on it, and they made it into a T-shirt. Obviously, it’s great when business owners can take those and put a positive spin on them or whatever catharses they need to. Like I said, it’s a hard human problem to deal with negative feedback. Obviously, when you put your heart and soul into your business, you don’t want to hear bad things about it.
Kaitlyn: I don’t do ratings anywhere unless I’m using a ride-share app, and then I automatically give five stars because I feel like there’s a difference in stakes. Even if I have a kind of a bad dinner should I be impacting someone’s livelihood over that? I don’t know. I certainly wouldn’t give anyone less than five stars on a ride-share app unless they physically accosted me or something. To me the difference in my stakes of spending $10 on a cheeseburger that wasn’t that good versus a small business owner’s stakes of...
Ashley: Their family’s livelihood.
Kaitlyn: Being able to pay for their lease. To me, I can’t imagine the incentive to write a negative review unless something deeply, deeply horrifying happened.
That actually stops a lot of people from writing a review. We’ve done a lot of internal studies of both, what would stop someone who, say checked in at a business and didn’t write a review. So we know that they’ve been there and why haven’t they followed up and written a review? Or someone who’s written one or two reviews and then we don’t see them for a long time. This idea that’s like, “I haven’t had anything extraordinary happen to me at a business where I’d want to leave that review,” or “I had a bad experience, but I’d still like to go there again before I would ever write anything negative.” At least from the people that we survey, we know that that’s out there. Obviously, there are some negative reviews on Yelp, so maybe everyone doesn’t have that strong of a filter. But you’re not alone in this idea of wanting to be really, really sure before giving that feedback.
Kaitlyn: I’m curious how often you see people naming specific employees of a business in their reviews. I worked in a mall food court for five years. It would be my worst nightmare for my boss to be like, “Oh, this Yelp review says Kaitlyn was rude,” or something. It’s also impossible, like what am I going to do? Call that person back to the mall food court and talk it out in front of him? I don’t know that seems like a risky situation to me, too.
For me, anecdotally, most of the time people that are calling people out in reviews, it seems to be that they’re trying to reward them. They loved their server. They had a great experience with someone at a local business and so they want to give props or sort of reward that person. Obviously, that’s really great to see and if you are an employee hopefully you like seeing your name in that context, but yeah the negative context would be more unfortunate.
Ashley: So generally, how do you feel review apps, just reviews in general. The fact that anyone can review anyone else for a lot of different things has changed the dynamic between consumers and business owners. Do you like that new dynamic, or how do you feel toward it, or how do you think it’s changed?
I think the dynamic has increased consumer’s, as a whole, ability to trust unproven things so brands, were incredibly important, are still incredibly important, but with review sites, be them Yelp or TripAdvisor, or Amazon, I think people can, ideally, trust the underlying product a lot more when they see it does have a huge positive response behind it. So, as a consumer, I’m more likely to be more adventurous in my eating habits because I don’t have to be as concerned going in, “Am I going to have a good experience at this new restaurant?” When I’m purchasing something online that I can’t feel, or touch, or maybe it’s one of a bunch of products, at least I know that enough people have found it useful.
I think also the metadata that comes along with reviews is really important and something where Yelp shines as well. I use this feature called Search within Reviews a lot on Yelp because I have a one-year-old now, and I like to get a sense of how other parents feel about it, so I can actually search for the word kids or toddler on a restaurant and hear from reviewers who brought their kids, and so they can talk about the kids menu or things like that. So even just beyond just the star rating, I’m able to go much, much deeper just because there’s this open text review product now that’s out there for businesses and for products and all the like.
On the whole, I do think that it’s really increased the transparency and also the adventurousness of the common consumer.