Immersing listeners in 3D sound

Advances in recording technology mean that listeners have become used to sounds giving them the notion that they are ‘inside’ an experience. But theories and practical advances from the Applied Psychoacoustics Laboratory (APL) at the University of Huddersfield mean that music, games and films are becoming more immersive than ever.

Dr Hyunkook Lee’s work is helping to add a crucial dimension to the soundscape – height. Reflections in venues like London’s Royal Albert Hall mean that the audience hears sound from above as well as around them. But recreating that experience via a recording, heard through loudspeakers or headphones, has been elusive until now.

3D sound is beginning to give a more complete sound picture that places the listener within an event – whether it’s a recording of a concert, or a digitally-created environment in Virtual Reality (VR). “It’s tied to psychoacoustics – how we perceive sound,” says Hyunkook, who joined the University in 2010. “My research is three-dimensional sound recording and reproduction - recreating or representing the acoustics of a real space in reproduction.”

Hyunkook’s ideas are being translated into cutting-edge microphones that capture ambience as well as the source of sounds, and software that can recreate a truly immersive aural experience. Schoeps, one of the world’s leading microphone companies, took Hyunkook’s findings on board in producing the ORTF 3D, an innovative new microphone array.

Four pairs of small microphones are arranged precisely so that one microphone per pair points upwards to capture the ambience from above, while the other points in the opposite direction to record source sound from floor level. Hyunkook’s research provided the theoretical basis for determining the correct angle and spacing between the microphones.

"We’re recognised as one of the leaders for expertise in 3D audio recording and reproduction"

It’s been used at a variety of high profile events, such as the French Open tennis championships, and at the BBC Proms to capture the famed ambience of the Royal Albert Hall. In both cases the sound from above gave a 3D depth to the listening experience. “In classical recording you capture the room, as well as the source,” Hyunkook declares. “It’s a purist approach.” VR and gaming having also benefitted the technology, with the sound given by its wider ‘sweet spot’ making gamers feel they are really ‘there’ in the virtual world.

In addition to his ideas being adopted for equipment, Hyunkook’s work in software that accurately represents a sound that envelops the listener from all directions is also making an impact. 5.1 surround sound has given cinema goers, or home cinema systems enthusiasts, the feeling that sound is around them for many years.

But as Hyunkook points out, “Surround misses the height element. We are developing new algorithms to render surround sound with height. There are a lot of fundamental questions to answer within this – like, how do we perceive spatial impression vertically? This has not been done much before. Previous research was all about sound coming from the horizontal, or in the single plane in front of you. But our research at the APL deals with sounds arriving from lots of different locations in space, to model the perceptual mechanism of human spatial hearing more precisely.”

Recent research on the virtual elevation effect by Hyunkook has also led to the development of software that enables listeners to perceive auditory height without having speakers actually set higher up. This will have useful practical applications in home and headphone listening environments.

"Our research at the APL deals with sounds arriving from lots of different locations in space"

Vital to Hyunkook’s research is the Critical Listening Room, built in 2014 through a grant from the EPSRC and support from the School of Computing and Engineering, and housed in the Technology Building. Up to 24 speakers can be set up to cover all currently available 3D sound formats, including Dolby Atmos, Auro-3D, DTS-X and 22.2. Some are near to ground level, and other speakers hang above the listener, who truly feels they are ‘inside’ the music when hearing a 3D recording of Bach’s ‘Toccata and fugue’ recorded on the 19^th century organ in Huddersfield’s Town Hall.

As a recording engineer himself, he is very active in producing practical audio content to help people experience what immersive 3D sound can offer. For example, he has recently recorded a London-based choir Siglo de Oro’s Pure Audio Blu-Ray album “Missa Tulerunt Dominum Meum” in Dolby-Atmos and Auro-3D formats, which delivers a true representation of the beautiful acoustics of the Merton College Chapel in Oxford.

“The University already had a good reputation around music and audio technology, and we have developed a name for ourselves over the last five years in 3D audio,” says Hyunkook. “We have focused on aligning ourselves with the Audio Engineering Society as it is the biggest community dedicated to audio. We’ve regularly presented papers and spoken at its conventions, and at the most recent one in Japan in August 2018 we had a lot of interest from Japanese gaming companies. We’re recognised as one of the leading places for expertise in 3D audio recording and reproduction.”

Partners

Fraunhofer IIS, Volvo Cars, L-ISA, Schoeps, Genelec, New York University, USA, KAIST, South Korea, Chalmers University, Sweden