Reimagining Realism in Video Games & Movies

Hideo Kojima, the  Japanese video game designer, screenwriter, director, and producer talks about realism in the Video Game and Movie industry.

—
October 15, 2017

Apes learn to speak, gain human intelligence and rise in arms against humanity. It’s an absurd premise made even more ridiculous by the fact that the militant apes in War of the Planet of the Apes are nothing more than the creation of computer graphics.

Despite the wrong setting and computer graphic characters, the result is a powerful movie with evocative characters that brilliantly exposes the mechanisms of conflicts and wars happening in our world today.

The movie is a masterpiece, proving how useful fiction can be to convey deep feelings, empathy and truth. At the same time, the film and its use of computer graphics raise an interesting question: What defines realism in entertainment – namely in movies and games – and what are the methods to achieve it?

From Makeup to Made Up
Both movies and video games rely on technology to try and capture a sense of reality and convey it to viewers or players.

While masterpieces such as Planet of the Apes continue to appear from time to time, in recent years Hollywood has leaned heavily on the “Based on a True Story” premise. These days many movies tote slogans such as “An Inspiring True Story,” and this is not limited to stories of human drama; you’ll even find the caption “Inspired by True Events” plastered on horror movies as well.

One of the causes of this phenomena may be the changes in our environment. Social networks and YouTube are flooded with real videos taken by people from all corners of the globe. You can find anything from videos of daily life, funny moments and breathtaking landscapes to the imagery of accidents and disasters exceeding anything previously shown on television news. All of this near instantly, right there on your smartphone or tablet. Consequently, even Hollywood, the land of entertainment, has been unable to escape the influence of realism.

One reflection of this influence can be seen in the found footage movie trend, such as Hardcore Henry, Cloverfield, REC, all of which use first person cameras and footage from security cameras. Another apparent influence comes from the massive success of first-person shooter games. The first-person perspective initially introduced in games as a means to increase the player’s sense of presence is now being adapted to film. This tendency toward realism can be seen in other blockbuster movies, sometimes using additional methodologies.

Filmmakers strive to achieve realism in their work by having performers partake in real action scenes on set, performing wild stunts, employing massive amounts of extras, crafting life-size sets indistinguishable from the real thing and more. For example, in The Force Awakens, life-size Millennium Falcon and X-Wing props were created for the performers (and similarly for Episode 8: The Last Jedi).

It is understandable that some media and critics want to emphasize the return to analog technologies when faced with the success of video games and widespread use of computer graphics in movies. It’s reminiscent of how the introduction of TV and later video games, were postured as a threat to film. New hardware and new technologies are prone to the opposition.

Whether a movie feels real or not is up to the audience. But because movies are fiction, to begin with, it’s odd to discuss how a film is real because it was shot with analog methods, or how a video is artificial and doesn’t feel real due to its use of computer graphics.

The Planet of the Apes reboot, beginning with Rise of the Planet of the Apes, brought a unique perspective to this argument, efficiently rebooting the concept of movie making via state of the art digital technologies. Not only that but as mentioned earlier, it proved the power of stories to convey truth through fiction. The story, portrayed with both computer-generated apes and real humans, touched our hearts and made the whole debate of computer graphics versus live action analog seem irrelevant. It showed us once again that whether it be computer graphics or analog methods, both are merely a means to achieve realism in film.

Planet of the Apes (1968) is based on the novel of the same name by French author Pierre Boulle and is said to be based on his experiences as a prisoner of war during WWII. It portrays a reversal of roles in which apes, seen as inferior to humans, overtake humanity, and in doing so upsets the racism and minority suppression associated with the white supremacist ideals of Western culture.

Planet of the Apes didn’t depict reality at face value but instead took full advantage of science fiction’s ability to use metaphor and philosophical experimentation to dig deep below the surface. An exercise which brought it a remarkable success, and spawned a total of five films.

The 1968 version of Planet of the Apes also made use of the most advanced technologies of the day to portray apes that have the intelligence and a culture exceeding humans. Those techniques were analog, namely special effects makeup and costumes.

The success of the movie, however, is not the result of technology alone. Much of the credit goes to its self-conscious depiction of social problems such as racism, nuclear weapons and cold war through the fiction of evolved apes and atrophied humans. On the topic of racism specifically, the film dives into themes of language (humanity in the film can’t speak English) and genetics.

The series was rebooted as a trilogy in 2011 starting with Rise of the Planet of the Apes, and while they are original stories inspired by the classic Escape from the Planet of the Apes, Conquest of the Planet of the Apes andBattle for the Planet of the Apes, they faithfully inherit the spirit of the original movies. That is to say; they continue to use the latest technology to convincingly portray the fiction of evolved apes, as well as tackle issues in society today. The second movie of the trilogy, Dawn of the Planet of the Apes (2014) and the latest War for the Planet of the Apes (2017) not only preserve that spirit but have consistently improved upon it.

Where lead makeup artist John Chambers and his team helped transform actors into apes in the original movies, performance capture did the work in the modern take on the films. In the performance captures, the performances, movements and facial expressions of a performer in a real-world, three-dimensional space are turned into digital data that can later be manipulated and reproduced with computer graphics.

The result of those captures is entirely fictional apes with highly realistic speech and emotion. These nonexistent apes have achieved a level of realism light years beyond their former SFX makeup and costume iterations. From their movements, expressions, and silhouettes to the way individual hairs are affected by muscle and wind simulation, the level of detail is astounding.

Digital technology brings the clash between warring ape and human factions to life on the screen. With the absence of state-run military forces; US Military, British Military, German Military, etc., in the films, we experience what is akin to “real” modern-day conflict; bloodshed that knows no nationality or borders. What’s more, gene therapy medication created by humans is presented as the root catalyst for the rise of apes and downfall of man – an irony that does not escape the viewer. (The previous series used the Sci-Fi gimmick of time travel as the cause.) The Planet of the Apes trilogy tells the story of “Our Planet” in the here and now all too convincingly.

Digital Worlds for Digital Characters
This level of realism and power of persuasion could not have been achieved without the latest digital technology. Technological advancements have continued to push for an ever more convincing movie reality. Analog or digital, that is not the crux of the issue.

No matter the type of technology used, analog or digital, the purpose of films is to achieve an unprecedented realism in fiction.

The same is true for the interactive fiction of games. Digital technology must be used to provide users with an experience, story, and message. And as with cinema, technology in game development has evolved. Once characters were represented by basic shapes and symbols, but now photogrammetry 3D scanning and performance capture allow real-world performances and movements of actors and actresses to be faithfully recreated in the digital realm. Physical-based rendering also introduced natural lighting. Because of this the background and prop creation process for video games is now shifting toward the creation of, and then scanning of real-world objects into the game. This move toward analog realism, using real performances and creating authentic props, is the same process movies now employ.

In my current project, DEATH STRANDING, we’ve captured real, analog actors Norman Reedus and Mads Mikkelsen, and are recreating them digitally within the game. The ability to control a real actor is unique to the fiction of games, and leads to a more realistic experience, and that is the shared aim of games and movies alike. Digital technologies are the magic ingredient to achieve this goal.

In that regard James Cameron’s influence on the world of games and movies is striking. His drive to develop innovative technologies in service of narrative realism is unmatched. From the revolutionary CG morphing technology of Terminator 2 to the development of his very own 3D camera for Avatar‘s 3D imagery. Not stopping there, for the upcoming Avatar sequels, he’s announced his ambition to develop glasses-free 3D viewing technology. Thanks to the advances made by pioneers like James Cameron, digital technologies have become a bridge that continues to bring cinema and games ever closer.

Virtual Reality
Virtual reality technology is a unique example of technology that frees both games and film from the frame that binds them. Whether created via an analog process or interspersed with digital technology, until film and games are released from the confines of the screen, they will be bounded by a frame. It isn’t enough to draw the viewer in with a strong feeling of presence and engrossing performances; they still can’t enter the scene. When all is said and done, the viewer is restricted to peering into a flat, framed screen of one kind or another.

Many movie creators have surely felt restricted by this dilemma. A notion backed up by the fact that several accomplished directors are starting to show interest in VR works. Kathryn Bigelow premiered a short VR film at this year’s Tribeca Film Festival (2017), Alejandro González Iñárritu followed suit with his short film at Cannes, and Ridley Scott has set up a VR division within his production company. These directors are utilizing the magic of digital technology to create realistic experiences beyond the boundaries of cinema, the boundaries of the frame. VR is posed to change established genres, such as fiction, documentaries, and art; to meld together and thus transform the viewer’s perception of realism.

VR documentaries, in particular, may be the first to transform the concept of a viewer’s expectations of realism in a video. VR, with its frameless, 360-degree presentation allows the viewer to experience events and stories with unparalleled realism. Viewers no longer see people, battlefields, and events within a frame, but are instead engrossed in an atmosphere that brings them face-to-face with those people and takes them directly inside the place where it happened. This new perspective is sure to impact fiction as well.

But unlike movies, games are an interactive medium. Combined with VR, games hold the potential to create an entirely new form of entertainment, something that defies the definition of film and games.

 

By Hideo Kojima

Read the original article at Rollingstone.com