Not just wrinkles, technology to rejuvenate actors
One of the works that appeared in the end of 2019 by Netflix, online video viewing service, is "Irishman" (original title "The IRISHMAN"), directed by Director Martin Scoses.This work is a multi -polar development that is screened not only as an online viewing program distributed on Netflix, but also in a single -building movie theater.Based on a work called "I Heard You Paint Houses" released by novelist Charles Brandt in 2004, it is a masterpiece gang movie depicting the United States from the 1950s to the 1970s.
"Irishman" Final Trailer (Netflix Japan)
The topic of "Irishman" was the characters played by two leading actors, Robert de Niro (born 1943, 76 years old) and Al Pachino (born 1940, 79 years old).It is that the acting and video of the young age appeared very naturally.* Actual shooting is from August 2017 to March 2018
In the video, each actor appears in the youngest appearance of 24 years old, and also appeared in the appearance of 36, 41, 42, 47, and 55 years old.Of course, I didn't divert the old video, nor was a young actor of a similar substitute.Also, not all virtual actors made in CG played.The reality is expressed by the special effects of CG/VFX, which takes advantage of the performance of old -fashioned actors.
『アイリッシュマン』の特殊効果は映画『スターウォーズ』のCG/VFXでも知られるILM(Industrial Light & Magic) 社が中心となり数社のCGプロダクションが担当しました。ILMの中でも古株にあたるVFXスーパーバイザー、『インデペンデンス・デイ』（1996年）『ザ・マミー/呪われた砂漠の王女』（2017年）など数多くの作品のVFXを担当したPablo Helman氏が『アイリッシュマン』のVFXチームを率いました。
ILM's "Irishman" introduction page https: // www.ILM.com/vfx/the-irishman/
一般的に若返り映像というと、シワ取りやシミの除去、美白効果や、たるんだ顔の輪郭の補正など、プリクラのような補正が思い浮かびます。けれどもILMでは単なるシワ取りや肌をなめらかにするだけではないアプローチに取り組みました。ILMのR&Dチームは、過去の実写映像を機械学習し、俳優が若返った顔を生み出す、FLUX(Face Lux)と呼ばれるツールを2年がかかりで開発しました。
The results were accepted by the strict eyes of directors and actors, highly evaluated and used for video production.The fact that this rejuvenation video was realized by modern technology was that the two leading actors have been active as actors since they were young, and many movies and many video data existed.
"Irishman" VFX making (Netflix Official)
What is the de-aging technology used in "Irishman"?
The idea of the DE-AGING tools began to go back around 2015, when Pablo Helman was in charge of Martin Scorsese's film "Silence" (original title: "Silence") released in 2016.。The request of Scorsese there was "keeping technology away from actors."VFX's face reproduction, special effects, and synthesis is one of the difficult expressions to approach real human beings even if you use the current state -of -the -art technology.
As a matter of fact, a special shooting environment that hinder actors' performances, such as drawing dots on the face with special markers, wearing special cameras for shooting faces, and blue and green backgrounds.It is common to record data for VFX.At the time of 2015, there was no technology to fulfill the request of Scorsese, so the film planning of Irishman was not realized, but it was postponed.
In order to realize the concept of de-aging, the experiment was conducted first.In order to get the accurate data of the face without a special marker, three cameras were usually used.The three cameras that were synchronized have acquired accurate three -dimensional data in the sole of the triangle.In other words, if you shoot from three places at the same time, you need 3x3 = 9 cameras.A camera for acquiring a geometry (shape) of the face on the left and right of the RED camera for film video shooting was installed, and a terrible equipment called "THREE HEADED MONSTER" was used.
‘THE IRISHMAN 'AND THE‘ THREE-HEADED MONSTER' (from Associated Press)
At the time of the test, all three were RGB cameras, but at the time of actual operation, two of them on the left and right were not affected by lighting, and the three -dimensional distance used for automatic driving of LIDAR, such as LIDAR.The camera was used.As a basic technology for performance capture, the door to CG VOL.DISNEY RESEARCH ZURICH MEDUSA, door to CG, introduced in 4.The LightStage introduced in 9 was used.
A few seconds of prototype video was made over several weeks.At that time, the 1990 movie "Good Fellows" featuring De Niro appeared as a material to show 76 -year -old De Niro in his forties.What was revealed in this prototype was not the face of De Niro, a few decades ago, but a few decades ago of the characters appearing in "Irishman".That is.It is difficult for actual people to have a beautiful and youthful face with beautiful skin, but old actors say, "This is an active actor for 30 years.I was pleased.
Based on such a situation, LEANDRO ESTEBECORENA, an ILM association vfx supervisor, gathered images from the age of 30 to 55, and databized several thousands of shots (scenes).Around the eyes (eyebrows, wrinkles, eyelids), where the age of the age appears remarkably (eyebrows, eyelids), the face, nose, mouth, etc. when sweating.
As a result, instead of a general face, based on the unique habit and expression of the actor, we searched for a video similar to the shots of an old -fashioned actor who took a live -action picture from the database when the actor was young and learned machine learning.It was used as a original data to replace the video.This tool was called "AI face Finder".In addition to the face, the neck (the neck) and hands are also the part where the age appears, and VFX was also used there.
In the 209-minute movie "Irishman", the effect of this de-aging was used 1,750 shots.It is a very large amount of special effects that are not movies that sell the CG/VFX effect, such as science fiction movies depicting the future.And for those who are watching the work, it is the most important factor that the actor's true acting is enjoyable, regardless of whether it is a video made of CG/VFX.For a long time, CG/VFX technology has not been able to catch up with the ideal, and works have finally been realized, and Martin Scorsese and appearing actors seem to be very satisfied.
Utilization of AI of Netflix mainstream business and AI research by Netflix Research
I mentioned earlier in the use of VFX and AI in Netflix's distribution works.
Although it does not appear much, Netflix uses machine learning technology for the recommendation function and personalization function that recommends recommended works for viewers.The recommendation is based on various viewing data from Netflix viewers.
As an analysis element, which video content did you watch to the end, or did you not see it until the end?If it was a feature or a series of series, how much did you watch?Which device did you watch on TV or smartphones, tablets, and PC?Do you watch during the day, evening, night, holidays, or per day?It is information such as.
On the other hand, data that is not used in Netflix includes information on gender, age, and country (region).This is because each person's preference is different, and Netflix is considered that it is not possible to categorize likes and dislikes in gender, age, and where they live.In addition, the data on the content side includes the director, actor, script, story complexity, the number of scenes, the location of the location, and the movement of the video.
The accumulated data will be used as information on the selection of actors and directors when producing Netflix's original program.For example, if you are a horror movie -loving viewer, you will consider planning a video work based on a survey of what actors, what directors, what kind of story (place) you like.Furthermore, in order to introduce what kind of budget to spend on what kind of situation, data is handled with data, such as which videos from videos are appropriate to cut out of static image thumbnails, and machine learning is used.I am.
Not only the general -grained images that everyone watches by the persuasive data analysis, but also the creation of a niche program such as a specific genre and a specific viewer enthusiastically accepts it based on data.It is thought that it can be expanded.Netflix considers that these machine learning platforms can be used across the organization, so it is not just knowledge and experience, not just knowledge and experience, using marketing, content development, advertising, and buying programs.We are promoted to make a basis.
・ Numerous research themes by Netflix Research Institute: https: // Research.Netflix.com/articles
・ Netflix Research: Machine Learning: https: // www.YouTube.com/watch? v = x9zes-fsxgu
・ Netflix Research: Machine Learning Platform: https: // www.YouTube.Com/Watch? v = SJ7F-38UV-4
AI is a technology to fulfill its purpose
"Irishman" was initially worked by a major US -a -Japanese movie distribution company, but it is a work that says that Netflix has been invested and continued to make the distribution right because the production costs were too expanded.
As you know about the AI utilization approach and strategy in Netflix's video works this time, you may have noticed that you are there.That is, "AI" is not on the front, but "AI" is being used as the best means at the moment to achieve the purpose you envisioned.If you use AI in a true sense, "AI" is not the subject of "in AI" or "AI", "AI ....I do not emphasize.In the future, if better technology and technology can be used, it is likely that they will use them to promote new things.
Future schedule of this series: "The door to CG" is a little different from the mere AI topic, and we will cut out topics from the context of CG/VFX and art.Please look forward to the use of AI in the field of video production, advanced tools that have increased value with AI, topics that make you feel the potential in the future, and technology.If you have any theme or wishes you want to take up, please let us know the editorial department.
Contributor: Koo Ando