Blog

AR

Viewing posts tagged AR

Double Nomination at Raindance Film Festival

We are delighted to see “Rise of Animals with David Attenborough” Magic Leap experience being nominated for two awards at Raindance 2019. It was a great privilege to work on interactive & spatial sound design alongside talented creative and technical teams.

Through the power of hand interaction you’ll discover and bring to life some of the first creatures that roamed the earth – all within your own environment using Magic Leap technology. Driven by a voiceover from David Attenborough and using ultra-realistic real-time animation sequences, this powerful Mixed Reality experience on Magic Leap One is both educational and entertaining.

Raindajnce Festival – https://www.raindanceimmersive.com/#/rise-of-animals-with-david-attenborough/

1.618 Digital Team

Immersive Audio Podcast Episode 28 – Audio For Augmented Reality

Summary

In this episode of Immersive Audio Podcast, Felix Thompson, with the help of insightful guest speakers, will look at Audio for Augmented Reality, and how it’s changing peoples’ lives today.

Augmented reality, or AR for short, is an industry that’s been silently booming over the last couple of years. Unlike virtual reality which totally immerses the user, AR offers a blend of the physical and the virtual worlds.

With the huge success of Pokemon Go, which allows people all across the world to see Pokemon floating around in real life via their mobile devices, augmented reality developers want to take the next step in revolutionising how we hear the world too, making augmented reality experiences for tourism, education, music – and even reinventing hearing aids.

 

Listen to Podcast

 

Guest Speakers

Elliot Graves, Atlantic Productions

Elliot leads Atlantic Productions’ Immersive and Digital content. As an immersive experience Director, he works across all XR technologies including augmented reality, virtual reality and AI empowered experiences. Prior to Atlantic, he worked in the agency space, creating immersive projects across the world for Nike, Google, Ford, Unicef and UEFA. Elliot started his career in factual production, working with The Royal Geographical Society, the United Nations and the BBC across a range of different initiatives.

 

Jean-Marc Jot, Magic Leap

Dr. Jean-Marc Jot joined Magic Leap to start up its audio algorithms team. Prior to this, Jot led DTS audio processing R&D and innovation. He had also led the development of SoundBlaster audio processing algorithms and architectures, including EAX and OpenAL technologies for game 3D audio authoring and rendering. Before relocating to California in the late 90s, he conducted research at IRCAM in Paris, where he designed the Spat software suite for spatial audio creation, performance and virtual reality.

Immersive Audio Podcast – Episode 22 Audio Design For Mixed Reality Panel at GDC feat. Jean-Marc Jot

 

Sofia Brazzola, Sennheiser

Sofia Brazzola is the User Experience Manager with the AMBEO team based at Sennheiser, where she leads user-centric research to validate and develop new solutions for a range of 3D audio applications. As part of the R&D team who developed the “AMBEO VR Mic”, she launched Sennheiser’s first early-adopters program by engaging with a community of beta-testers and content creators. She is currently leading Sennheiser’s Augmented Audio Developers Program, where early-adopters engagement, user-centric innovation and strategic marketing play another important role in AMBEO’s thought-leadership in AR audio.

Immersive Audio Podcast Episode 23 Sofia Brazzola

 

Kedar Shashidhar, Magic Leap

Kedar Shashidhar is an audio generalist focused on new realities. With a focus in creative research and development, he has worked to help develop new tools, create content ,and explore the boundaries of audio in new realities. Most notably he has contributed to the development of various audio toolsets including Magic Leap, Steam Audio, and Ossic. He also formed an indie game studio, Blackout VR, focused on developing audio only games. Kedar is also an active voice for audio in new realities having previously spoken at AES, Game Sound Con, Indiecade, and more. Currently at Magic Leap, Kedar works on bringing the newest innovations in mixed reality audio to content creators in order to continue pushing the boundaries of what is possible for audio in new realities. You can find his latest work in Magic Leaps Soundfield Audio toolset and a number of experiences on the Magic Leap platform.

Luis Marcelino, Evollu

Luis Marcelino is CTO at Evollu and an adjunct professor at the Polytechnic Institute of Leiria. Evollu aims to help companies in the field of hearing rehabilitation increase their sales and expand their business. They use hearing tests and adaptive listening to support people in the process of improving their hearing. They have developed an augmented audio app that gives people “enhanced hearing.” This “enhanced hearing “ will allow users to selectively improve the quality of sound from different sources. This app was made for people with mild and moderate hearing loss, or for environments where it’s difficult to hear.

 

Per Beck Hansen, AWE

Per Beck Hansen is co-founder and CEO of AWE, a Danish startup working with audio development and design for interactive VR/AR. They have created projects from international VR installations to music driven health care solutions to augmented audio games for tourism. As part of the Ambeo development programme, AWE created the app 1807 for tourists in Copenhagen. Spatialized audio for visitors to Copenhagen – alternative to a tour guide. Users get to experience Copenhagen as it was 200 years ago, while under siege from the British. Expect guns and grenades.

 

Marc Green, University of York Audio Lab

Marc Green is a third-year PhD student at the University of York AudioLab. His research is focused on environmental soundscapes, with a view to utilising techniques from machine listening and spatial audio in their analysis. Using Sennheiser AMBEO technology, he has developed an AR app that could assist urban planners, researchers and architects in creating more positive acoustic environments for people to live and work in.

 

Damian Murphy, University of York

Damian Murphy is a Professor of Sound and Music Computing in the Department of Electronic Engineering at the University of York, where he has been a member of the academic staff since 2000. On top of that, he is the University Research Champion for Creativity. Damian was key in making links between the Sennheiser AMBEO programme and researchers at the University AudioLab to explore the potential for the creative use of augmented audio as part of our wider immersive audio research. One particular area of application is in the use of machine learning to understand and discern more information about our sound environment, and how augmenting this soundscape with additional audio content might change our perception of this soundscape. Damian’s PhD student Marc Green is currently leading on research in the area of spatial audio, machine learning and environmental soundscape and decided to look to how AMBEO technology might feature in his work.

Immersive Audio Podcast Episode 8 Damian Murphy

 

Eric Tarr, Belmont University

Dr. Eric Tarr is an associate professor of Audio Engineering Technology at Belmont University in Nashville, TN. He teaches classes on digital audio, computer programming, signal processing, and analysis. He received a Ph.D., M.S., and B.S. in Electrical and Computer Engineering from the Ohio State University. He received a B.A in Mathematics and a minor in Music from Capital University in Columbus, OH. He released an iOS app as a development partner with Sennheiser Electronic using the Smart AMBEO headset for VR/AR audio. The app simulates multiple types of hearing loss for use in hearing science research. It is called the Impaired Hearing Simulator (IHSIM) and is available for free from the iOS app store. He has developed software with several other audio companies including Apogee Electronics, Skywalker Sound, L.R. Baggs, and Harrison Consoles. His research has spanned across the topics of speech signal processing, musical robotics, sound spatialization, acoustic and electronic system modeling, hearing loss, perception and cognition.

Show Notes

Website for awe: https://theawe.dk/

Info on Microsoft Hololens: https://www.microsoft.com/en-us/hololens

More information about Pokemon Go: https://www.pokemon.com/uk/pokemon-video-games/pokemon-go/

Article about an augmented reality try on app: https://www.realitytechnologies.com/try-on-shoes-using-this-augmented-reality-app-for-ios/

More information about Sennheiser Augmented Audio Lab and  AMBEO AR One: http://sennheiser-ambeo.com/arone/

More information about St Mary’s Abbey: Reconstructed: https://www.york.ac.uk/news-and-events/events/yornight/2015/activities/yorkshire-museum/hearing-the-past-st-marys-abbey/

More information about Magic Leap’s Wearable 3D Audio: https://www.magicleap.com/news/news/wearable-3d-audio

Information about Eric Tarr’s work with Sennheiser AMBEO: http://sennheiser-ambeo.com/community/eric-tarr/

Information about the textbook written by Eric Tarr: https://www.mathworks.com/academia/books/hack-audio-tarr.html

Article about Evollu’s work: https://www.jornaldeleiria.pt/detalhe-noticia-print/8844

More information about Luis Marcelino’s work with Sennheiser AMBEO: http://sennheiser-ambeo.com/community/luis-marcelino/

Information of Sigur Ros and Magic Leap collaboration: https://pitchfork.com/news/sigur-ros-and-magic-leap-release-new-vr-experience-tonandi/

How to Balance VR and Realism: https://uploadvr.com/balance-gameplay-realism-creating-vr-sim-experience/

Creadits

This episode was produced by Felix Thompson and Oliver Kadel with the help of Abbigayle Bircham and Share O’Hare and included music by Knobs Bergamo, Isaac Joel and Yung Koolade.

Immersive Audio Podcast Episode 26 – Tom Szirtes (Mbryonic)

Summary

In this episode of the Immersive Audio Podcast, Oliver Kadel sits down with Tom Szirtes, the founder of Mbryonic.

Mbryonic is a London-based digital studio that create cutting edge virtual reality, augmented reality, mixed reality, and other digital content. They work with a wide variety of companies from Amazon to Essex Council to Gazelli Art House. Tom founded Mbryonic with a simple mission, to apply his creative and technical skillset to innovate business and artistic applications of technology.

Before Mbryonic, Tom has had years of experience as a lead developer and manager in the AAA gaming and telecommunications industry working for companies like Sega Europe, Square Enix and Orange.

Alongside this career, Szirtes is an established music producer, DJ and label owner. His music output spans over a decade, some fifty records and numerous international performances. This creativity is at the heart of Mbryonic as well. For Szirtes, the technology always serves creativity not the other way around.

On this episode, Oliver and Tom discuss the ways that virtual reality technology could enhance the experience of listening to music.

Listen to Podcast

Show Notes

Mbryonic Home page: https://mbryonic.com/

Shur-I-Kan Information on Free Range Records website: http://www.freerangerecords.co.uk/artists/view/Shur-I-Kan/

Shur-I-Kan on Bandcamp: https://shur-i-kan.bandcamp.com/

Tom Szirtes on LinkedIn: https://www.linkedin.com/in/tomszirtes?originalSubdomain=uk

More Information About The Process of Creating Play Sage Gateshead: https://mbryonic.com/portfolio/play-sage-gateshead/

Play Sage Gateshead, The Finished Product: http://playsagegateshead.com/

Mbryonic’s work with Essex Council: https://mbryonic.com/portfolio/delivr-it-logistics/

Information about Amplify VR: https://mbryonic.com/portfolio/amplify-vr/

Video of Shur-I-Kan’s Minecraft set: https://www.youtube.com/watch?v=xx5F3tDxPyM

Information About a Similar Minecraft Live Music Event: https://cuppawow.com/coalchella-minecraft-music-festival/

More Information about Innovate UK’s “IC Tomorrow” Competition: https://www.roadtovr.com/innovate-uk-awards-210k-to-vr-and-ar-pioneers/

More Information about Mbryonic’s Virtual Reality Music Experiences in Issue 12 of Techmutiny (interview located on page 22 of the document): http://techmutiny.co.uk/attachments/article/49/TECHMUTINY_Issue%2012_FINAL020316.pdf

Creadits

This episode was produced by Abbigayle Bircham and Oliver Kadel with the help of Share O’Hare and included music by Knobs Bergamo.

Immersive Audio Podcast – Episode 23 Sofia Brazzola (Ambeo Sennheiser)

Summary

For this episode of the Immersive Audio Podcast, Oliver Kadel is joined by Sofia Brazzola, the User Experience Manager with the AMBEO team based at Sennheiser.

Sofia holds a Master in Design with a focus on Research and Visual Communication, and in her professional experience at Sennheiser has honed music and technology innovation expertise. Sofia is the User Experience Manager at Sennheiser’s AMBEO team, where she leads user-centric research to validate and develop new solutions for a range of 3D audio applications.

As part of the R&D team who developed the “AMBEO VR Mic”, she launched Sennheiser’s first early-adopters program by engaging with a community of beta-testers and content creators. She is currently leading Sennheiser’s Augmented Audio Developers Program, where early-adopters engagement, user-centric innovation and strategic marketing play another important role in AMBEO’s thought-leadership in AR audio.

Today, Oliver and Sofia discuss the role of the AMBEO team, Sennheiser’s AMBEO line of products, and the Augmented Audio Developers Program.

Listen to Podcast

Show Notes

Sennheiser: https://en-uk.sennheiser.com/

AMBEO: http://sennheiser-ambeo.com/

Magic Leap: https://www.magicleap.com/

AMBEO Lab App: https://itunes.apple.com/us/app/sennheiser-ambeo-smartheadset/id1231509105?mt=8

HD25 Headphones: https://en-uk.sennheiser.com/on-ear-dj-headphone-hd25

HD600 Headphones: https://en-uk.sennheiser.com/best-audio-headphones-high-end-stereo-hifi-hd-600

AMBEO VR Microphone: https://en-uk.sennheiser.com/microphone-3d-audio-ambeo-vr-mic

AMBEO Smart Headset: https://en-uk.sennheiser.com/finalstop

AMBEO AR ONE: http://sennheiser-ambeo.com/arone/

Augmented Audio Developers Program: http://sennheiser-ambeo.com/developers-program/

iOS: https://en.wikipedia.org/wiki/IOS

Unity: https://unity3d.com/

Comply Memory Foam Ear Tips: https://www.complyfoam.com/

Credits

This episode was produced by Abbigayle Bircham, Oliver Kadel with the help of Kaelyn Tindall and included music by Knobs Bergamo.

 

Immersive Audio Podcast – Episode 19 Adam Levenson (Gaudio Lab)

Summary

For this episode of the Immersive Audio Podcast, Oliver Kadel is joined via Skype by Adam Levenson, the Vice President of Business Development at Gaudio Lab.

Based in Seoul, Gaudio Lab develops audio technology solutions for VR and streaming media including the Sol VR360 SDK and the Sol Loudness SDK available for licensing now. Working with the likes of Honda Innovation and Naver Corporation, Gaudio Lab received the “Innovative VR Company of the Year” award from the AMD Studios VR Awards in 2017.

Adam Levenson has 25 years of experience in audio production and technology. During his tenure as Senior Director at Activision, Adam established the Central Audio team supporting work on major franchises such as Call of Duty, Skylanders, James Bond, Spider-Man, and Transformers. He’s since worked in high-level positions at companies like Somatone Interactive, CRI Middleware, and Krotos.

On this episode, Oliver and Adam discuss the topics of loudness, audio quality in streaming and standards for creators.

Listen to Podcast

Shownotes

Gaudio Lab Homepage: https://www.gaudiolab.com/

MPEG Wikipedia Page: https://en.wikipedia.org/wiki/Moving_Picture_Experts_Group

SBS Korean Broadcasting Company: https://www.sbs.co.kr/

Honda Innovations: https://www.hondainnovations.com/

13th Floor Productions: https://www.13thfloorentertainment.com/

5G Case Study: https://www.gaudiolab.com/thirteenth-floor-and-g-audio-form-alliance-to-provide-a-cinematic-vr-platform-for-5g-mobile-networks/

Activision: https://www.activision.com/

Antoni Gaudí: https://en.wikipedia.org/wiki/Antoni_Gaud%C3%AD

SDK: https://en.wikipedia.org/wiki/Software_development_kit

Sol VR360 SDK: https://www.gaudiolab.com/sol-vr360-sdk/

Pro Tools: https://www.avid.com/pro-tools

Sol Loudness SDK: https://www.gaudiolab.com/sol-loudness-sdk/

Netflix: https://www.netflix.com/

Amazon Prime: https://www.amazon.co.uk/Prime-Video/b?ie=UTF8&node=3280626031

FCC Calm Act: https://www.fcc.gov/media/policy/loud-commercials

Pandora: https://www.pandora.com/

Spotify: https://www.spotify.com/uk/

Twitch: https://www.twitch.tv/

Immersive Audio Podcast – Episode 4 Sally-Anne Kellaway

Summary

In today’s episode we are joined by Sally-Anne Kellaway via Skype. Sally is the Senior Audio Designer on the Microsoft Mixed Reality – Audio and AI team, and an industry leading VR Audio Evangelist.

She has extensive experience in sound recording, editing, implementation and testing for interactive media in standard and VR/AR and has an interest in implementing true 3D spatial hearing and binaural sound in video games and other interactive experiences. She is the founder of the Virtual Reality Content Creators Network of Australia (VRCC)  which is a not for profit community for virtual reality and associated industries. The VRCC advocates for innovation and connection for all professionals working in virtual, augmented and mixed reality industries in Australia.

Previously the Creative Director at OSSIC, the leading audio technology providers active in the VR space, she has a Masters in Acoustics with a focus on psychoacoustics and spatial audio from the University of Sydney. Sally-Anne is sought after by VR and AR developers, Software and Middleware developers, Conferences, Meetup Groups and Panels to develop and consult on sound design, implementation, tutorials and presentations. She is the Founder of the Women/NB in VR Group for Australia and co-organiser of the San Diego VR community group and is on the AES Technical Committee for Game Audio (Spatial Audio) and on the Diversity and Inclusion Working group.

Sally is a vocal advocate for VR, AR and MR as the future of audio, entertainment and industry and uses her education in Sound Design, Acoustics and Psychoacoustics to rise the tide for all audio professionals passionate about the future.

In this episode, Sally-anne speaks about head-tracking, the future of Audio AR, and diversity in the industry.

Listen To Podcast

Shownotes

Sally-anne Kellaway: http://soundsbysal.weebly.com

University of Sydney: https://sydney.edu.au/

The Virtual Barbershop: https://www.youtube.com/watch?v=IUDTlvagjJA

OSSIC: https://www.ossic.com/

Audio Engineering Society: http://www.aes.org/

Virtual Reality Content Creators of Australia: http://www.vrcc-aus.rocks/

The Wave VR: http://thewavevr.com/

Zero Latency VR: https://www.zerolatencyvr.com/

1.618 Digital to join the panel at the Immerse UK event

Welcome to the Real World – Immersive Audio for Augmented and Virtual Reality

1.618 will be taking part in a panel discussion with the UK’s leading Immersive Audio experts. Join us as we talk about the latest cutting-edge work in immersive audio research, development and content design.

The event will cover everything from audio AR/VR creative process, application workflows and the potential for product development. This guarantees to be a truly interactive event, with teams demonstrating some of the most exciting immersive audio technology and experiences in the field.

Other speakers include:

  • Jelle van Mourik (Sony) – Immersive Audio in Games
  • Gavin Kearney (University of York) – Immersive Audio Research
  • Gareth Llewellyn (Mixed Immersion) – Immersive Audio Production for VR
  • Chris Pike (BBC) – Immersive Audio in Broadcast

The panel discussion will take place at 5.45pm-6.45pm on Wednesday 24th May 2017. Click here to register and get your free ticket: Ticket Registration

For more upcoming events hosted by Innovate UK click here

See you there!
1.618 Digital Team

1.618 to join the panel at The BVE 2017 Excel London

1.618 will be taking part in a panel of 3D audio experts at the BVE 2017 Expo Excel London on 2nd March. Join us as we discuss the next generation of immersive audio, covering everything from future technologies to next generation audio. We’d love to see you there!

Discussion topics will include:

Insights into the current and future technologies that will be taking the industry by storm

The role of object-based audio – how this will improve the customer experience.

Binaural audio – creating a really immersive sound for your audience

Will next generation audio take off – how far away are we?

The panel discussion takes place at 12.30-13.30 on Thursday 2nd March. You can get your free ticket here: Ticket Registration 

Check out what else is taking place at BVE on Thursday 2nd March here

See you soon!

1.618 Team

 

Discussing the importance of spatial audio on happyfinish.com

Importance of Spatial Audio In VR Content

Hearing is fundamental to our perception of the surrounding world.  Achieving this effect in virtual reality requires audio that sounds real and authentic.  Implementing spatial audio to create full immersion in 360° video or interactive VR requires capturing audio or a physical acoustic modeling of the space where the scene takes place.  An appropriate soundscape can provide the quickest path to immersion for just about any VR experience, and even removing the visual element, still enables us to sufficiently perceive the surrounding world – giving us a sense of space, time, and presence.  In contrast, the silent experiences, or the ones with incongruent sound would break the sense of presence and immersion, thus immediately removing the suspension of disbelief, and as a result substantially degrading the overall experience.

Spatial sound recording or let’s do it in post?

The use of conventional industry formats such as Mono (single channel) and Stereo (two channels) are a basic requirement, although they are limiting and no longer sufficient to offer full immersion in 360° videos or interactive VR experiences.  The use of spatial audio is the only way to create true three-dimensional audio, which utilises higher number of channels, be it capturing sound on location or through the means of sound design and mixing in post-production.  Depending on the nature of the project both methods are important.  Often to design the full sonic experience in VR, it requires spatial sound recording on set along with sound design and spatialisation of individual elements in post-production such as atmosphere, dialogue, foley, sound effects, and music.

Ambisonic format is the most effective method to capture location sound

There are a number of ways to capture the location sound.  However, the most effective method is to record in an ambisonic format which utilises four channels capturing the sound in all directions, along with discrete sound sources such as dialogue or any required diegetic sounds that are part of the scene.  The latter can then be positioned accordingly within the 3D soundfield by employing specialist spatialisation software within audio editing application or a game engine.  This approach enables VR audio content makers to work with an adequate resolution within the virtual space for positioning sound components across four, 16 or more virtual or physical channels.

Ambisonic sound offers a number of significant benefits that play a crucial role in making experiences as realistic as possible.

-Firstly, sound that was captured in all directions then enables the user to move their head and body while wearing any head-mounted display, and with a use of head-tracking system perceive their own dynamic position within the space in relation to the surrounding environment.

-Second, greater channel count offers more accuracy in positioning individual elements within the 3D space. This avoids everything coming from the same general direction as is common when listening to music, but lacking in realism when comes to creating a metaverse or offering your audience an authentic 360° video experience.

Why is this essential?

The considerations mentioned above are essential due the phenomenon described as a head-related transfer function (HRTF), which is a response that informs how our ears perceive sound from own position in space.  Collectively, head-related functions for both ears give a perception of binaural sound, enabling us to effectively identify a location and a distance of sound sources by constantly receiving sonic information to measure sound intensity and the time difference between sounds arriving to both ears.  We re-create this psychoacoustic process in post-production to then achieve a latency-free, real-time binaural rendering via a close approximation of personal HRTF.  It is essential to take human physiology into consideration when making audience fully immersed and enjoy their experience, be it a story, game or cognitive therapy etc.

The use of audio in marketing campaigns to guide your audience in 360° content

Unlike 2D content where a viewer can see the entire field of view in one direction, the 360° environment presents challenges as well as the opportunities for creative content makers when it comes to constructing the narrative.  Spatial audio can be an effective tool to lead or surprise your audience.  By looking in one direction at any given time, the viewer can easily miss out on what is behind them or sidewards, by implementing sonic cues within the space we can control or suggest a narrative.  By helping our audience to navigate through their point of view, we can ultimately guide them to and encourage them to engage with a specific element within the experience.

What is more important?

When combined effectively, fully integrated visual and sonic perception work in perfect harmony that enables us to see, hear, feel and appreciate the beauty and richness of our world.  Virtual reality already proved its effectiveness in video storytelling, gaming, educational training, social interaction and medical applications.  In order to make any of the above experiences successful, it requires a coherent approach of applying sound and visual content to make it as effective for its purpose as possible – more immersive, more authentic and as the result more engaging, more memorable, more empathetic, more fun and ultimately good enough to have a desire to come back and experience it again and again.

1.618 Team