At Oculus Connect conference yesterday, we got a glimpse of what Facebook was after when it got into virtual reality: a VR environment where you can feel like you’re hanging out in the same physical space as your friends, complete with realistic body language and emotional responses. Facebook CEO Mark Zuckerberg appeared in virtual reality with some of his colleagues to play cards, travel to a simulation of Mars, and hold a video call with his wife.

One of those colleagues was Michael Booth, Facebook’s product manager for social VR. Booth comes from the world of game development, but what he’s helping to build now isn’t just a Facebook-owned massive multiplayer VR game — it’s a new place that’s eventually supposed to connect to every part of the social network, for all its billion-plus users. After the keynote, we asked him a few questions about how that might work.

Interview has been condensed and lightly edited for clarity.

Can you tell me more about the demo’s emotion system?

There’s no built-in eye tracking, there’s no special hardware involved. What we were doing there, basically, I would describe as VR emojis. So by doing certain gestures, we can trigger different emotions. For example, if I want to look surprised, I can take my hands with my Touch controllers, put them on the sides of my cheeks, and my avatar will go ‘Whoahhh!’ [Booth makes surprised face.] I can put them on top of my head, and my avatar will be like, ‘Eeeeh!’ It’ll be scared, right?

Now, what we actually did during the demo was slightly different, because I think there’ll end up being two tiers of how this will work. If you’re a professional entertainer and you’re trying to put on a show, you’ll have a little more complicated puppeteering controls. But for the general public, having these sorts of gestures that are intuitive, kind of make you into a vaudevillian actor, that seems approachable. But this is experimental; this might all change.

Would people be able to make their own gestures, like making a sticker pack?

That is a very interesting question. The challenge with this is, these avatars are animated three-dimensional objects, so they’re fairly complicated to create. But something that’s certainly possible — again, this is speculative — say we have six slots and we have a whole bunch of emotions you can pick from, and you decide which emotions to put in each of those slots. So you can sort of personalize your personality.

A lot of non-VR Facebook is asynchronous, so you’re creating and curating this feed of your life instead of interacting live. Is there a version of that in virtual reality?

We have been thinking a lot about that. We don’t have anything specific that we’re ready to share about that, but it’s a very interesting and challenging problem. I think the way that we’re approaching it initially is, we are working on ways to connect people who don’t have VR with people that have VR on Facebook. You saw that on the demo with the Messenger call — so any of your friends could use Messenger and do a video call with you while you’re in VR, and you’ll see them.

Similarly, we’re working on ways to record video and share that with your feed, so your friends on Facebook can see what you and your friends did in VR, so you become a superstar on Facebook. You’re the one with VR, so you can create all this cool stuff and share it with your friends.

Will this system work in Gear VR, even though it doesn’t have something like Touch?

Right now, we’re focused on Rift and Touch. We’re trying to allow you to connect with your friends, regardless of where they are in the world, and feel like you’re literally sitting there in person with them, and can reach right out and touch them.

So that’s the core of where we’re coming from. There’s so many different variables, so we’re focusing on Rift and Touch right now, but we are Facebook, so we do want to be on all the platforms. We want to support Gear VR, we want to be on the Vive, on Sony Playstation, whatever interesting VR platforms are out there.

After Oculus was bought by Facebook, Mark Zuckerberg was talking about how you might be able to go to events and feel like you were right there with your friends. Is that part of this now?

What we’re focusing on right now is just connecting with your friends and sharing content in different ways, streaming things in and out of Facebook. But that’s version one. We have lots of ideas, that being one of them, to explore in the future. There’s lots of interest in attending interesting events virtually.

Facebook does a lot of work trying to make sure people who have very little access to the internet can get into Facebook, but this seems like it’s much more at the other end, where it’s extremely exclusive. How do you approach that gap?

Again, it’s these connections. When you’re in VR you can interact with people that are in regular 2D Facebook and vice versa, and it gives us the network effect, it gives people ways to connect back and forth. There’s a whole continuum of content there that you can share and transfer.

I look at this as, this is back in the bad old days of giant cellphones that required battery packs in your car, right? "That’s really cool, you can make a call from anywhere, but it’s huge and ugly and I don’t want that in my car." Eventually we’re going to get to a place where VR is this, right? [Points to glasses.] It’s as just as awesome as cellphones are.

How permeable is the barrier between the non-VR Facebook app and the in-VR stuff?

That is a wide-open area of research right now. Facebook is surprisingly complicated. There are lots of things you can do on Facebook — you can post and share and cross-post and comment and share the cross-posting comment, and how we present that in VR is a challenge. The ultimate goal is, everything you can do on Facebook you should be able to do in VR in some fashion. But we’re taking baby steps first, to try to get the most important content that makes the most sense in VR.