Tons of Iwata Asks: Wii U GamePad details

Nintendo published the second edition of its Iwata Asks Wii U series. The latest discussion focuses entirely on the GamePad.

There are still lots of technical tidbits that will probably go over most people’s heads (including mine!), but there is a bunch of interesting information. The latter parts tackle the GamePad design, the changes it experienced since it was first showed at E3 2011, and more.

The summary is below. For the full Iwata Asks, visit this link.

The Wireless System Challenge

– Yamashita worked on the GamePad firmware and software used to connect it to the console
– GamePad doesn’t directly process games
– Signals are sent from different input devices
– The console then performs the processing
– GamePad finally shows images sent from the console
– Ito was involved with the overall system design of the hardware
– One of Ito’s tasks was to make sure there wasn’t a noticeable delay with the sending/receiving of images and sounds between the GamePad and console
– Mae worked on the wireless communication technology from the GamePad to the Wii U
– Ibuki worked on the industrial design of the GamePad
– The entire industrial design group finalized the design
– Ibuki also worked on the overall hardware
– Ibuki made a ton of design elements
– Some objects were actually made by hand
– Iwamoto worked on pulling together development of the software for controlling the GamePad
– He also determined what functions to give the controller and the SDK development environment
– Time different and language barrier were challenges for Iwamoto
– There were phone meetings every day and team members traveled back and forth
– Iwamoto noted that a lot of talent has gathered in America
– Iwata: “I truly think that the professionals in each field in America are highly specialized.”
– Iwata: “With ordinary software that lacks communication features, the environment doesn’t affect it, and you can create the data with the premise that they will be correctly retrieved all the time. You can’t see the actual wireless communication, and each environment is different. Also, while a certain amount of data gets garbled, it still needs to runs well. These are the characteristics completely different from ordinary software.”
– It was a challenge dealing with wireless communication element changes for each environment and the fact that there are drastic changes occurring all the time in video game images (at the same time)
– The fact that the GamePad is a wireless system was an important task
– Images had to be compressed and sent in real time
– These were two key areas
– Mae was worried about latency
– Iwata: “With the usual wireless video transfer methods, even if a slight latency occurred, it was okay as long as it didn’t get stuck along the way. So with ordinary video playback, the system would buffer a certain amount of data before it plays in the case where data isn’t being sent on a consistent and constant basis so the video can play smoothly. With the Wii U GamePad, however, Mario has to jump as soon as you press the button, so if there’s latency, it’s fatal for the game. You had to take on a challenge that no one else had before.”

One Solution for Multiple Issues

– The team worked with a number of companies when it came to the wireless system and latency
– MegaChips worked with Nintendo up to the very end, from IC design to the development of the firmware that would run on the IC
– Ito: “But since that method would cause latency, this time, we thought of a way to take one image and break it down into pieces of smaller images. We thought that maybe we could reduce the amount of delay in sending one screen if we dealt in those smaller images from output from the Wii U console GPU on through compression, wireless transfer, and display on the LCD monitor”
– There was a good feeling from the start when this was discussed
– Ito: “Generally, compression for a single screen can be done per a 16×16 macroblock. And on Wii U, it rapidly compress the data, and the moment the data has built up to a packet-size that can be sent, it sends it to the Wii U GamePad.”
– Wireless side was hard since data rapidly came in small portions, making it difficult to put it together and deliver it in real time while minimizing errors
– Iwamoto: “Usually in image compression and wireless communication, it allow for latency to maintain quality. If we tightened up on latency, even if it ran well under the best conditions, it was difficult to address problems—for example, sending certain images when signals are weak—that arose in specific situations.”
– When conditions aren’t constant, it makes wireless communication difficult
– From a wireless standpoint, it’s tough to have players hold the GamePad and move it due to the Doppler effect
– Iwata: “To address why holding it vertically may have been an issue, is because radio waves don’t spread well underwater, but the human body is 60 to 70 percent water, so it can interfere with radio wave paths. Since you hold it both vertically and horizontally, it gets more difficult to place the antenna where the radio waves can easily reach it. If the cost wasn’t an issue, there would be many ways to resolve it like adding more antennas. Besides, since you make various movements with the Wii U GamePad, the radio waves might get distorted. You had to think about all those problems.”
– There will be differences in the GamePad’s wireless reach for walls depending on if the house is made of wood, concrete, materials inside, etc.
– Iwamoto: “However, if you place the Wii U console in something like a metal TV stand it may deflect the radio waves thus reducing its usable range. Radio waves weakens by the square of the distance, so even within the same space, too much distance could make them weaker, and having obstacles in between would be a disadvantage.”
– Iwata could use the GamePad in his bathroom since it’s separated by a single wall from the living room

“It Comes Down to Guts!”

– One big problem was the sending after compression for computer graphics
– CG is different from natural images
– More bitrate was needed than originally thought
– Early CG sample used was primitive; crisp without gradation; detailed and difficult to compress
– The sample was initially just a cube moving around
– Next sample was a grid-shaped pattern that enlarged and rushed at you
– Iwata: “In the world of video compression technology, there are such things as types of video that are more ideal to compress, and those that aren’t well suited. You faced one of the most difficult things in a sample right around the start and were like, “Oh no! This is a disaster!””
– This was a little before E3 2011
– There was an emergency meeting with Ito, Mae, and Yamashita, wondering what they would do about this
– Ultimately Yamashita feels it was good that they found such a difficult video when they did
– Tuning was done for around a year
– Test sample is even being used now
– The team scrambled to improve image quality just before E3 2011
– There were a lot of phone meeting to figure out how to improve image quality and bitrate
– Yamashita: “I can’t forget them either! (laughs) When we ran software on the Wii U GamePad, some problems arose that were different than the case with the sample CG. When we displayed images meant for the TV on the LCD monitor on the Wii U GamePad, noise specific to compression technology was appearing during fade-in and fade-out. So at E3, Ito-san and I were running around the development teams that were working on exhibit titles.”
– Some games had less noise, others caught them off guard
– Something with a lot of small characters appearing like Pikmin went okay
– There was a lot of noise in Super Mario when coins came rushing out
– This surprised Iwamoto
– Nintendo partnered with Broadcam, a leading wireless chip vendor, for wireless tech.
– Challenges were overcome by using the latest tech. and capabilities unique to Broadcam
– Takeda said “We need people with guts” regarding partners

“We Gotta Make a Controller!”

– Tea table was overturned three or four times for the GamePad design
– Circle Pads changed to analogue sticks that can be pushed down as additional buttons
– GamePad also gained grips
– There were a number of big design changes for operability in a short time
– For the design shown at E3, the design group was of a mind to make a pad-like device
– Concept was a device that would be worthy of being in the living room
– Surface was very uncluttered initially
– The team realized it was difficult to use after E3
– NES version of Mario Bros. was played on the GamePad
– Team couldn’t play it very well
– Iwata remembers people saying that the GamePad was tiring and difficult to play
– When they decided they wanted to do something about the GamePad, they heard about desire within the company to change the specs to improve the controls
– Ibuki: “We would have regretted releasing it like that. So we started from the beginning to see whether a flat, pad like design, or a shape of a traditional controller that players could firmly grab that emphasized on controllability was the right way to go. We held a lot of meetings within the team over what the Wii U GamePad should be like. A lot of opinions came up.”
– Iwata and Miyamoto wanted change to happen
– GamePad changed so your fingers wouldn’t get tired
– Still a long ways to go
– Took time balancing the controller’s ease of use with a design appropriate for the living room
– Grips weren’t finalized until the very end
– Initially looked at a more flat, square shaped pad to a version with gradual grips and considered different options
– There was an in-house survey as to which was easier to use
– Votes were divided, so they couldn’t decide
– There were a lot of tests until it reached its current form
– Ibuki: “To be honest, we didn’t know how tacking grips on the back of a square pad-like device would make it any easier to hold. So we made a bunch of designs, I carved them by hand, adjusted them with clay, did that day after day for a long time, went to have it checked by those involved, and got told it was difficult to hold!”
– They said it wasn’t pretty good, so they repeated that process
– This was the hardest time for Ibuki
– Ibuki spent a lot of energy making it lighter
– They were able to chip away a gram or two here and there by doing things like messing around with the chassis, used to protect the screen
– Nintendo made it as light as possible without causing durability problems
– When someone made a suggestion about changing the design, they had to know how many grams would be involved
– They wanted the GamePad to weigh roughly 500 grams
– When they were near 500 and some grams, all suggestions that may put more weight on were refused
– Iwata noted that Nintendo tests a lot with a controller or handheld in terms of weight, but they set records this time

A More Attractive TV

– NFC came in half-way through
– Yamashita said that they didn’t expect the GamePad to have so many functions
– Software had to be made for NFC, TV control button, and the geomagnetic sensor
– Iwata announced NFC at a corporate strategy meeting in January, but the development hadn’t caught up yet
– Wireless radio waves are usually sent by wire to avoid interference during development of the GamePad
– Yamashita tried a test at home and was surprised that so much graphics showed up even when not connected by cable
– He was surprised at how it was wireless
– He also thinks other players will be surprised at such a basic function of the system as how the graphics rendered by the console appear on the GamePad
– Ito thinks tech-savvy people will be surprised by the lack of latency
– Iwamoto hopes people notice how it has the functions of a portable but controllers usually don’t
– Nintendo added the capability to send video data from the camera onto the existing wireless communication IC for sending game visuals
– Iwata was thinking about how hard Iwamoto had been working after seeing the time it was taking to make the camera run well
– Camera was abandoned for a bit at the beginning of development
– Yamashita: “At the beginning of development, we thought the schedule was going to be tight, and we abandoned the camera for a spell. But then talk of at least being able to manage still images or at least 5 frames per second (fps) started bubbling up, and eventually that became a desire for 30fps, so the camera returned right away. (laughs)”
– They had been at a loss until E3 2011
– They then pounded it out last year from summer to fall and made it work
– Camera has to perform compression and decompression twice to perform a wireless transmission
– When you draw something on the GamePad, the touch input data goes to the console and then returns as images after processing
– Iwata: “People with a little technological knowledge will now be able to understand a little more deeply the value of what this team has done. At the present stage, I sense two sides to the Wii U GamePad. One is that this is the first home game console that allows one person to play video games while someone else watches television. The other is how the television becomes even more attractive when you use the television and Wii U GamePad as a set. It only became that way, however, after the incredibly gritty and persistent hard work of the people who made the Wii U GamePad. It’s interesting how a single genius didn’t solve all the problems, which makes for renewed and deeper understanding of Takeda-san’s statement “In the end, it comes down to guts!” In the end, it really was about guts. (laughs) Thank you for today.”

Thanks to 4Him for the tip.