News On Japan

Japanese and English Speakers Watch a Talking Face Differently

When a friend leans across a loud izakaya table to tell you something, where do your eyes go?

Japanese and English Speakers Watch a Talking Face Differently

Most people would say "to the face." But a face is a big place, and the part you look at may depend on the language you grew up with.

For anyone who makes talking videos for viewers in Japan and abroad, this small habit matters. A tour guide who recorded a welcome message in Japanese may want an English version too. With an ai lip sync video, her face can deliver that new line, her lips moving in time with a recorded English voice track. Whether viewers even look at those lips is a question researchers in Japan have put to the test.

The Mouth Quietly Helps the Ear

We tend to think of listening as a job for the ears alone. In practice, the eyes pitch in. On a busy train platform or in a crowded bar, a glance at someone's lips can help you catch words the noise would otherwise swallow.

The best-known example of this teamwork is a lab trick called the McGurk effect. A recording of someone saying "ba" is paired with video of a mouth saying "ga." Many listeners then report hearing a third sound, such as "da." The ears and eyes disagree, and the brain settles the argument by blending them.

This is where Japan comes in. Earlier studies found that the illusion is much weaker for Japanese listeners than for English listeners. Something about how Japanese speakers combine sight and sound seemed to be different, but the reason was not clear.

What the Kumamoto Team Measured

To look closer, researchers at Kumamoto University and their colleagues ran four experiments with young adult native speakers of English and Japanese. Volunteers watched short clips of two women, one Japanese and one English speaker, saying the syllables "ba" and "ga." They pressed a button to say which one they heard. The team timed their answers, recorded their brain waves and tracked where their eyes went.

The results pointed in opposite directions. For the English speakers, seeing the matching mouth movement shortened their response time. For the Japanese speakers, the researchers observed the opposite effect. Adding the moving mouth made them slower, not faster, and the brain-wave readings followed the same pattern.

The eye tracking, done with 13 English speakers and 16 Japanese speakers, helped explain why. The English group's gaze was drawn to the mouth, especially in the moment before any sound came out. The Japanese group showed no such pull. The authors suggest the English speakers used those early lip movements to get ready for the sound. The Japanese speakers did not.

It helps to keep the scale in mind. These were small groups of young adults naming single syllables in a lab, not friends chatting over dinner. The study does not show that Japanese people ignore lips in daily life. It shows a clear difference in how two groups used the mouth during one simple task.

A Habit Learned Early, Not a Switch

One of the most interesting results came last. The team asked some Japanese volunteers to focus on the speaker's mouth on purpose. They did look at it longer. But looking longer did not make the mouth change what they heard.

So the difference is not only about where the eyes happen to land. The authors point to language as one possible reason. Earlier research they cite sorted English consonants into five or six groups that lipreaders can tell apart. Japanese consonants fall into only three. If the mouth carries less useful information in a language, children who grow up with it may learn to lean on the voice.

That fits another earlier finding the paper describes. At age six, Japanese and English speaking children were swayed by the mouth to a similar, small degree. After that, the pull grew with age for English speakers but stayed the same for Japanese speakers.

Culture may also play a part. The paper mentions studies of facial expressions in which Eastern viewers paid more attention to the eyes and Western viewers to the mouth. The authors are careful to say their experiments were not designed to prove a cause.

Masks, Subtitles and Anime Voices

Seen through this research, a few everyday things in Japan look a little different. Face masks are a common sight during cold and hay fever seasons. They cover the very part of the face many English speakers watch. It is tempting to connect the two, but the study did not test masks, so any link is only a guess.

Movies offer another angle. Foreign films in Japanese cinemas are often released in two versions, one subtitled and one dubbed, and viewers choose the one they like. Anime has its own tradition, too. Voice actors commonly record their lines while watching the animation. They fit their timing to mouths that often use just a few simple shapes.

Audiences everywhere enjoy both, which is a useful reminder. People adapt to many kinds of mouth-and-voice matching, and what feels natural is partly learned.

Making Talking Videos for Both Audiences

If you make talking clips for viewers in Japan and in English-speaking countries, the research points to a few simple habits.

Put the audio first. If many Japanese viewers lean on the voice, a muddy recording will hurt more than a less-than-perfect picture. Record in a quiet room, speak clearly and keep music out from under the speech.

Keep the mouth timing natural anyway. English speakers in the study looked at the mouth early, before the first word. They may be quick to notice lips that run out of step. A front-facing shot with soft, even light makes the mouth easy to read for those who do watch it.

Add captions. Subtitles help both groups, along with anyone watching on a silent train. They also give viewers a second chance to catch a word that the audio or the mouth did not make clear.

Respect the person on screen. Only upload material you have the right and permission to use. If it includes another person's image or voice, make sure you have the consent required for your intended use.

Conclusion

Where our eyes go when someone speaks may be partly learned. In this study, English speakers were already watching the mouth before a sound came out, while Japanese speakers did not show that early focus on the lips. A good bilingual video leaves room for both.

News On Japan
POPULAR NEWS

A bear believed to be a mother that has been roaming residential neighborhoods in Maebashi, Gunma Prefecture, since November 17 was spotted again by a police officer early on November 21, prompting authorities to prepare for its capture as residents entered a three-day holiday weekend following days of repeated sightings, school closures and a frightening encounter involving a high school student.

Residents of Okinawa's remote islands are struggling to secure airline tickets as travelers seeking to accumulate loyalty points flood short-haul flights, with some boarding as many as 12 planes a day without stopping to sightsee, leaving essential routes fully booked and prompting calls for Japan Airlines (JAL) to reconsider its promotional campaign.

Traditional matchmaking is making a comeback in Japan as younger generations increasingly seek safer and more efficient ways to find marriage partners, with growing frustration over dating apps encouraging a return to arranged introductions once common among their parents and grandparents.

Nepalese police have detained 35 Japanese nationals and five Chinese nationals suspected of involvement in organized fraud operations at a mountain resort hotel near Kathmandu, where the group had allegedly established a sophisticated base equipped with a mock police station, photography studios and dozens of livestreaming booths.

A powerful surge of midwinter cold air is bringing heavy snow, strong winds and freezing temperatures to much of Japan ahead of the three-day holiday weekend beginning November 21st, with authorities warning of dangerous road conditions, possible landslides and traffic disruptions as snow and heavy rain continue along the Sea of Japan coast through the morning of November 22nd.

MEDIA CHANNELS
         

MORE Web3 NEWS

Artificial intelligence is transforming how people experience entertainment and digital media, with technologies that can change voices in real time, generate fragrances from musical notes and bring virtual characters into physical spaces taking center stage at Inter BEE 2026, one of Japan's largest media and entertainment technology exhibitions, which opened on November 18.

AI-generated images are becoming increasingly common across Japan, appearing on restaurant signs, food menus and even election posters, raising questions about authenticity, consumer trust and how far the technology should be allowed to replace traditional photography and design.

Surveillance cameras are entering a new era in which artificial intelligence can detect dangerous situations, alert authorities and even issue evacuation warnings automatically, with the latest technology showcased at an exhibition in Nagoya on November 18.

The cartoon-like promotions can be a creative way to promote your products instead of traditional product ads.

Kazuki Yokoyama, a 28-year-old entrepreneur from Niigata Prefecture, has been named among Forbes Japan's 30 Under 30 for 2026, earning recognition for an innovative business model that combines artificial intelligence with the acquisition and revitalization of small manufacturing companies struggling with labor shortages and a lack of successors.

Tokyo is preparing to introduce artificial intelligence-based traffic signal control across the metropolitan area as early as fiscal 2027, using technology that predicts congestion 30 minutes in advance and adjusts traffic light timing to improve traffic flow.

Election posters featuring AI-generated faces that did not resemble the actual candidates have sparked controversy in Amagasaki, Hyogo Prefecture, after a former prefectural assembly member acknowledged that four candidates in a November 15th city council by-election had used artificial images rather than their own photographs, exposing gaps in Japan's election laws and raising questions about the use of public funds.

A humanoid robot signed to the Japanese subsidiary of Elite World, one of the world's largest modeling agencies, made its runway debut at a fashion show in Tokyo on November 16th, walking alongside human models in a presentation exploring the coexistence of humans and robots.