General:Building the Interface of The Elder Scrolls: Blades in Landscape and Portrait

Building the Interface of The Elder Scrolls: Blades in Landscape and Portrait
(link 1 2)
Medium/Format	Live Presentation
Date	March 2019
Interviewee(s)	Marie Jasmin
Location	Moscone Center
Hosted By	Game Developers Conference

Good afternoon, UX summit. This is my first GDC talk, so whether you think it was terrible or decent, please fill out your forms; we want your feedback. Also mute your phones, please. My name is Marie Jasmin, I've been in game UI design for 10 years now and last year I joined a team at Bethesda Game Studios with an interesting mandate: to create an Elder Scrolls experience on your phone.

The vision for The Elder Scrolls: Blades was to create intricate role-playing game systems with depth set in environments that would push the edge of visual quality on the small devices while fully embracing portability and accessibility. Now it may not be apparent at a glance, but this vision for Blades contains a dilemma. Should the game be built in the portrait mode or in the landscape mode? Blades ended up being built for both and today we'll dig a little deeper into the vision that led us to choose dual-orientation support, then we'll look at the UI design process and how we made it happen, and finally we'll analyze the player response we have so far (considering we're still in early access - close beta, actually, but it's still interesting to look at today).

So, we're building an Elder Scrolls game on mobile. In essence, mobile gaming liberates the context of play away from the living room or home PC and takes it anywhere: on the couch, at the park, on the bus, at the airport, on the toilet, or during a meeting, or a GDC talk. Notice a few things here. In some of these contexts, people only have one free hand, so if gaming is to be made even possible there, portrait is a must. In fact, for some players, the one hand situation is a little more persistent. For others, it might be even permanent. In other contexts, the public eye is upon our player, and in those situations, portraits can act as some sort of a "stealth mode", where people may not necessarily appear to be playing a game. Our creative director, Todd Howard, affectionately calls portrait mode the "meeting mode". So looking at the context of play, we must conclude that if mobility is a requirement, then a portrait orientation is the only way to go.

However, remember that for Blades navigation in stunning environments is also a must. External research done on the way we hold our devices reveals that while portrait holds are relatively diverse - a lot of people hold their device with both hands or just one hand depending on the context - but the landscape hold is almost universal. People have both thumbs firmly placed on each side of the screen, towards the bottom. The physiognomy [sic] of people - or the ergonomy of the landscape hold - enables us to separate controls for camera and navigation in two. The alternative in portrait is the one finger tap-to-move, but this inevitably binds camera to the eyes, effectively reproducing the feeling of walking with a very stiff neck. And perhaps more importantly, navigation options for humans and Khajiit and Argonians alike (if you're familiar with the Elder Scrolls) are usually lateral. Above is mostly ceiling and sky, below is mostly ground and floor, not very interesting for navigation if you're a regular human, not Spider-Man.

So, in this example, you see two doors, right, so you could go left, you could go right, you have choices, you have agency. But when viewing this same environment in portrait, you can only see one door. So effectively what happens is that your navigation options are hidden under the fold - to borrow a term from web design. So basically, what it means is you need to scroll to be able to see your options. And this combined with the stiff neck phenomenon creates feelings of claustrophobia when navigating a 3D environment in portrait. Hence, we come to the ground-breaking revelation of this day at GDC: landscapes are better experienced in landscape. So let's do a little experiment. If you personally had to choose a device orientation to build The Elder Scrolls: Blades, which one would you go for? Those who vote for the portrait mode with better accessibility and mobility, raise your hands. Hm, a few people. And those who choose the more immersive mode... oh wow, that's interesting. A lot of immersive people here. Actually, this was an endless debate at Bethesda, so we took the opportunity here to turn lack of consensus into an opportunity for player agency. So we let people flip their phones whenever they please. And yes, we knew it would be expensive and painful and we had never done this before. What are we waiting for? Let's get started.

To begin with, we make design documents and menu flows and we make only one set of them. So to preserve our sanity, we go through the regular design process, but trying to keep the information presented to the players the exact same as much as possible, just in two different layouts, so a bit like mirror layouts. So what this means is that the UX or design team had minimal overhead for dual-orientation support, but the UI team needs to produce each mock-up or wireframe twice. On Blades, this added about 20-60% time to UI conception. It doesn't double the time because most visual communication problems have already been solved when you tackle the alternate mode. But the extra time invested in conception did yield some early benefits.

To provide an example, this is one of the earliest building menus we designed. And here I list all the materials people need to craft certain buildings, being as generous with information as I can so that players can make their own crafting decisions. And paper tests were relatively successful, so if I ask "Can you craft the smithy?" people would say "Um, no, because I'm missing a lumber and potatoes.". Relatively successful. But then came the time to adapt the layout to portrait, and while our list now nicely held a large number of items - and this was nice, we wanted to keep it this way, but there was no way all the information would ever fit the narrow space. So it forced us to adapt, right? So we started disclosing information depending on context. So here in the repair section of the Smithy I show the item condition and in the crafting section I show information about the materials required, but notice here that the materials information has been distilled to a red x or green check mark. And when we paper tested this again - the distilled version in portrait - the responses were noticeably more confident. "So can you craft a steel dagger?" "Sure, but it's not worth it. It has bad stats." So it was way better. So we realized that the compromises we had made for portrait were actually way more efficient, so we adopted them for landscape as well. The takeaway here is that the extra constraints on the UI team allowed Blades to identify certain issues early on and it saved us a lot of time down the line.

Thank heavens it did, because down the line is where time became excessively scarce. The implementation was time consuming. First, dual-orientation still means each of the two layouts need to adapt to the various aspect ratios, notches, and punch holes of the mobile market today. So even if all our documents were produced in the 16:9 aspect ratio, they needed to stretch vertically on iPads and horizontally on larger, newer phones. So our integration notes got quite complex and our prefabs in Unity got even moreso. But the real dual-orientation party happened in quality assurance. Because remember that players not only own differently-shaped and sized sizes of devices, they are also free to rotate them whenever they damn well please. So QA, to test this, needed to flip their phones in every possible game situation: in the middle of a combat move, during a loading, during a server call, during animations, between a button press and a button release... yeah. Bug festival ensued. In this instance, the UI disappeared entirely. We fixed it, but fixing in dual-orientation is a tedious and delicate affair. Our systems were sometimes redundant. It was a case of our animation system, so keyframes are basically copy-pasted. That meant we needed to fix issues twice. And most of the times, they were instantiated so the same widget would be used in both orientations. But that was delicate, because it means fixing a bug in one mode might break it in the other mode. We did get the interface to work, though, and we got a first playable demo in the hands of players in dual-orientation as early as we could.

Getting the game in human hands as early as possible was crucial to the dev process of Blades. Don't try to retrofit dual-orientation. Do it from the start. You will uncover flaws in your designs and you need to have enough time ahead of you to course correct. To provide an example here, we have a shield icon smack in the middle of the screen in landscape, and it's a critical combat mechanic allowing players to protect themselves with their shield. The icon was terribly-positioned, and my punishment as a designer for not having noticed earlier was to be forced to stand completely silent as I watched players struggle, physically, to even reach it and die. One of the many ways I've killed people with UI. Hahaha. We fixed it, of course. We moved the shield and we added a few player options in the process so the left-handed could enjoy the game as much as anyone else. But our first hands-on demo also opened our minds to an opportunity we had overlooked.

With the two different screen modes we could cater to different playstyles. So as a design experiment, we began optimizing the landscape mode for immersion, allocating a good portion of the screen real estate to 3D models, the player's avatar, etc. And in the portrait mode, we got rid of all that. It became our gameplay-first mode. We even cluttered the portrait mode 3D model with extra non-essential information that was nonetheless useful. This is a worst-case scenario by the way, but the the takeaway here is that the diverging layouts allowed us to be a lot bolder with our design decisions since we were catering to a bit narrower subset of our players for each mode.

While it's still early to draw any conclusions as to whether this grand dual-orientation experiment was all worth it, the results we have so far are interesting. First, if we take a look at the freeform comment our closed beta players are sending us through a generic feedback email - so we give them a pop-up: "Thank you for playing Blades. Send your feedback here." and they do. And we witness something unprecedented in UI. Apart from bugs, the majority of UI mentions are... positive? From my past experience, what typically happens is, in freeform comment, is that UI doesn't even get noticed or... it gets noticed, but that's when people tell the developer what's wrong with it and how to fix it. What's interesting, though, is that landscape mode gets the usual treatment. People don't mention it. It's the portrait mode that gets all the love. Our players seem to perceive it as some sort of a bonus mode that we've generously built for them and landscape appears to be considered "default" or taken for granted. Digging a little further into our letters of portrait mode appreciation, we realized that the design assumptions we made about people with different playstyles choosing a mode or the other to best suit them were partly wrong. So we do have some players who stick to one mode, but quite a few of our playtesters have gone to great lengths to describe their use of the portrait mode, and it's all about the context of play. So more like what I talked about earlier, some of the associated keywords in the feedback associated to portrait are "going" and "walking". I know, "walking"! Don't play video games while walking, people.

And telemetry data sort of confirms this. For instance - and take these with a grain of salt; we're still in closed beta - but we clearly see, though, that the session length on average is much shorter in portrait. So people play less long, and overall it's slightly shorter in the day time when people are going about their daily business, versus longer sessions in landscape mostly past 8pm local time. Another curious finding from digging into device orientation telemetry data is the slow but gradual adoption of the landscape mode by the same players over time. So it's as if our players kind of naturally realize that the experience is a bit better in landscape and they kind of adopt it. However, the adoption is not global. People play the quests, dungeons, more in landscape over time, but in menus it remains around 65% portrait 45% landscape kind of consistently over time. So globally the trend we're observing is that many of our players will tend to go about their daily business in in their town in short daytime sessions in portrait. So they'll go get their gear repaired, temper a sword, brew a few potions, and then they'll come back later to the game in the evening for longer sessions: to explore dungeons, fight dragons, or explore the Abyss outside of town.

So in a way, dual-screen orientation support has positioned The Elder Scrolls: Blades in the best of both worlds. It remains portable and accessible throughout the day while making no concessions on the stunning RPG experience it provides. The grand takeaway here is that, if well planned - and I need to insist on this - if well planned and if done from the beginning, both technically and production-wise, the return on investment of portrait mode support on a 3D mobile experience is well worth the extra costs. The the extra design constraints actually help improve the experience and our data shows our players appreciate it and they use it when it suits them. If you'd like to try it for yourselves, you can download the game. It will be available this spring to all. It's free, give it a run and send us your feedback. This is a feedback address. If you want to follow me, I rant about UI, and if you witness your colleagues playing Blades in meetings, share your stories; I want to hear them. Thank you.

---

We do have a bit of time for questions...

About how many - I guess "iterations" did you go through for the UI when you started to realize "Hey, we can kind of put these things together. Like we can use check marks instead of red numbers." etc.

So it depends on which parts of the UI. Some were pretty good from the get-go, others were very much iterated on. The game - actually, it's interesting because it's a smaller team than i'm used to; I come from AAA and it changed radically throughout the dev process. I'll share this one of the things we discovered in our first playtests last year was that people were obsessed with cheese. And we were like "Oh my god, cheese is one of the Elder Scrolls' core pillars!" and people couldn't interact with cheese, they were very, really sad. So we added cheese and now you can eat cheese in The Elder Scrolls: Blades. But yeah, some parts got actually really, really changed radically. It depends on the the UI. Crafting is one of the ones that changed a lot okay.

Thank you.

---

Wonderful talk. I have a bit of a question, sort of in the theme of one of our earlier talks. In projects like this, one of the hardest things to do is to get buy-in. So when you're starting a project like this, generally speaking, you need some data to say "we can invest all of this time because this is how our players are going to feel". How did you tackle convincing your team and your company to be able to do this, invest in the time to be able to do in portrait and landscape.

That's an excellent question. And yes, it's true, you need buy-in. We were lucky because it was the other way around on our project. It's the creative director who believed in dual-orientation support for Blades and he needed to convince us because the doers were like "but it's gonna cost this much money!" and he convinced us.

Amazing.

But that's easier than the other way around, yes.

---

So when you have your phone normally, it's in portrait like that's how all the apps are displayed...

Yes.

When you boot up the app and you have the splash screen that usually sets the precedent for whether they want you to turn your phone or not, how did you make the decision to make whatever the default whether it was portrait or landscape?

So the default is whatever the player is booting in. And one of the things we're considering now is maybe tutorializing the fact that you can even flip your phone. We have some hardcore portrait users you saw, like the about 65% of people are in menus in portrait and we think maybe it's because some people don't even know they can flip their phone because it's not very common yet in North America. So we're considering it.

Thank you.

---

I have a question for smaller companies with smaller budget. For instance, is it an option to play the game in landscape and the menu in portrait? Is it something that you already - you have "yes it could be an option" or did you evaluate this approach on "is it a valid approach"?

That's interesting, but it would force users to flip their phones at certain times. I'm thinking, but I don't think I've ever seen this approach.

Me neither. It's great to do both, but let's say that is it something that you analyze doing play test or the player?

No, we wanted to give full control to the players. Yeah.

And don't you think that it could be an option or not at all?

I don't know, I think it might be frustrating to be forced to rotate your phone when you don't necessarily want to, but that's just a hunch because I don't have an example of a case where it's been done.

No, me neither. Thank you.

---

Just a quick question. Because you talk about the all-rotation scenario with your QA team, I'm just curious what was the process with that - or with them? Because I think they might have lost their minds at some level because there was so many scenarios. And the follow-up question on that is how much data did you get from the QA on the UI/UX perspective.

Great question. Yes, they needed a hug at some point. But yeah, for QA it's... yeah, I said so, they carry basically the heaviest part of the burden because they needed to test every possible case. For localization it was twice, so every every language needed to be tested twice. Not all the the possible aspect ratios because you just need to test your smallest possible aspect ratio and that's everything twice. But for testing flipping their phones and - they kind of liked it, though, because they're sadistic. Like "I flipped my phone in the middle of this". Yeah, button press and release was like "why would you do that..."

Because QA.

So yeah, the buy-in - again to come back to the buy-in - needs to be global, right, the QA needs to be aware we're doing this, we're in this together, and yes they need a full team of support.

---

My name is David. Do you have any data on whether it was like harder to play in portrait? You were talking about how you'd sort of have a tunnel vision effect.

Yes.

You can probably still get through the levels. Does it take longer to get through or more tries? Do you know this?

That's a very good question. It doesn't take longer to get through the same level. But yeah, it's mostly a question of how you feel through it. And we we know also - it's actually really hard to get any data on this because people are in different contexts. They don't have time, they're kind of busy in portrait, usually. They want to go faster.

I don't think you would notice it's harder if you're playing.

The landscape people will explore a bit more: go to the side path, discover the hidden treasure, so it's very different ways of playing the same game, actually. Unfortunately, we have not a lot of data because we're still in closed beta, but that's what we observe is people are a bit more in a hurry, and you saw the session length is much shorter in portrait. So it's just a different way to experience the same game.

Thank you.

---

Hi, I had a question. You mentioned at the end, you validated that there was a good return on investment in doing both. But I guess, what were you looking in terms of data to kind of validate that?

Thank you. So was it a good return on investment? What we are looking right now to say that is mostly the reception of the players and also the media. It's interesting, I didn't put it on my slides, but we even have headlines in our reviews that are like "wow Elder Scrolls: Blades is getting one thing right: the portrait mode" and we're like... "we're a headline"? Like UI is not used to get so much limelight. But that's also all the good press the people are grateful for the existence of the portrait mode. We think that's really well worth it. It's hard to calculate, isn't it, because we also don't know exactly how much it costed us. It's hard to evaluate because we did it from the get-go in both orientations. But yeah, we believe it was worth it.

Thank you.

Thank you. That's all the time we have. Thank you very much.