“Ray Tracing is commonly used for indirect lighting,” Kasparas Eidukonis, Senior Audio Programmer, says. “Like when you shine a light into a room – maybe it bleeds out through the door and highlights the areas outside, maybe it bounces on walls and creates different colors of light based on the materials it hits.”

“For audio, it’s essentially the same. Light can be described as a wave, just like sound, it just behaves differently when it hits a surface. We shoot out rays that bounce around and propagate sound. And the more rays we shoot out, the clearer an idea of what it should sound like we get.”

Light can be described as a wave, just like sound, it just behaves differently when it hits a surface.

Let’s imagine you sit opposite another person at a table. If you talk, you can clearly hear each other. The sound waves don’t have an obstacle that separates the two of you. Place a lamp on the table, the sound changes slightly. Flip the table over and suddenly you have something between you and the person on the other side, muffling sounds. Is there a door? Is it open? You can keep building on the scenario, and everything you add will bounce waves – or rays, in the case of Ray Tracing – back to your ears, all in different ways. That’s sound propagation.

Now teleport yourself to the middle of the Kinglor Forest. Consider all the trees, plants, rocks, animals, flowing water, birds, enemy bases, and machinery. Imagine what they do to the thousands of rays the game sends out every frame and what it means for the audio experience.

“It’s super important for immersion to get the sound propagation right,” Simon Stevnhoved, Senior Sound Designer, says. “We are evolutionarily trained to be aware of our surroundings, and our ears can pick up on the slightest changes in our environment. When sound propagation works well, you might not pick up on it consciously, but when something is off, it quickly becomes noticeable. You know instinctively that if you make a loud sound – like firing a gun next to a canyon – the sound will reflect a certain way.

We are evolutionarily trained to be aware of our surroundings.

The team started out by using propagation tech adapted from Tom Clancy’s The Division 2, such as the Slapback, Bubblespace, and Obstruction systems. But, as work on Avatar: Frontiers of Pandora progressed, they realized that they needed to both improve what they already had and build completely new tech for the game. While looking for ways to improve sound obstruction, GPU Ray Tracing was integrated into Snowdrop, and the audio team saw their opportunity to jump on board.

“The Division 2’s solution wasn’t built for a jungle environment,” Kasparas says. “It was designed for a city environment, with blocky buildings and props. In Avatar: Frontiers of Pandora, the geometry is much more varied, with trees and leaves and bushes.”

So, the team got to work on building a small prototype for raytraced sound obstruction, which worked well for walking around the Kinglor Forest, hiding behind trees and tree trunks, and listening to water sounds. They were happy with what they had but knew there were a lot of improvements that could be made. Some environments worked better than others, for example. The team ended up with a hybrid system, mainly relying on the Ray Tracing solution, but still utilizing hand placed data which was already available, such as volumes for inside spaces and doors.

“In the old system, we would have a custom-built solution for each type of environment, for each type of plant, rock, cliff, terrain,” Kasparas says. “The advantage we were seeing with the Ray Tracing solution is that we could do much more with the rays. We wouldn’t need to build those custom systems, instead it would see all those things naturally. We didn’t need to edit a rock and say, ‘this rock is solid’ or ‘this bush is semitransparent’ every time – naturally the bush will have gaps and the rays can pass through it better than the rock.”

Another bonus of utilizing Snowdrop’s Ray Tracing system as a basis was that the team could move the process from the processor (CPU) to the graphics card (GPU), allowing them to create and sample a lot more rays, faster. Moving parts of the process to a separate piece of hardware also frees up the CPU to do other things. And being released exclusively on the current generation of consoles – and PC – helped, as there’s suddenly a lot more memory and power to take advantage of.

For an early prototype of the system, the audio team tried a very realistic approach to sound propagation – trying to mimic how sound propagation works in the real world as closely as possible. While it created an impressive soundscape, it also became too overbearing to be useable in a game.

You still need to tell a story; you still need to understand gameplay. It needs to be approachable.

“You would end up with sensory overload,” Alex Riviere, Audio Director, says. “It’s a lot of sounds: from insects to birds, foliage rustling with the wind, cues for gameplay and navigation, to name a few. It became too much to take in and understand.  It sounded great, but any new player would be overwhelmed in that situation. So, we had to take a step back and iterate on the sound mix to make it less overbearing. You still need to tell a story; you still need to understand gameplay. It needs to be approachable.”

There’s still a lot of exploration left in the audio field. But now, we have the opportunity to enter the various vast (and virtual) environments and landscapes of Pandora and immerse ourselves in the diverse sounds of the flora and fauna you can only find there.

More from Snowdrop

• Shining a Light on Pandora, Snowdrop’s Ray Tracing
• Crafting Pandora’s Breathtaking Landscape, Snowdrop’s scattering system

Avatar: Frontiers of Pandora © 2023 20th Century Studios. Game Software excluding 20th Century Studios elements: © 2023 Ubisoft Entertainment. All Rights Reserved. Avatar: Frontiers of Pandora and the 20th Century Studios logo are trademarks of 20th Century Studios. Licensed to Ubisoft Entertainment by 20th Century Studios. Ubisoft and the Ubisoft logo are registered or unregistered trademarks of Ubisoft Entertainment in the U.S. and/or other countries.

One of the first things that Oleksandr Koshlo, Rendering Architect and Quentin Kuenlin, Senior Rendering Programmer on Snowdrop, mention when we sit down to talk lighting and Ray Tracing in Avatar: Frontiers of Pandora, is the scale of the project compared to other projects that Snowdrop has been used for previously. And it’s not just about the map of the game, but also how far away the horizon is for the players.

“It’s a huge world, which means that you can see far, especially from high vertical positions. This means that we need to support very distant rendering and we need to give it a lot of care,” explains Quentin.

Compared to The Division 2, which was also built on Snowdrop but before current gen enabled Ray Tracing, Avatar: Frontiers of Pandora saw a lot of work going into supporting different lighting effects from a high distance, which presents light sources that allows distance processing volumetric lighting.

We have been able to drastically improve some of the techniques that we had to forgo in previous productions.

“With Avatar: Frontiers of Pandora being a current gen title, we have been able to drastically improve some of the techniques that we had to forgo in previous productions. Now, we could implement Ray Tracing for both lighting and reflections, and that’s a huge change.”

According to both Oleksandr and Quentin, the change in lighting quality is drastic compared to previous titles built on the Snowdrop engine, making things in the world much better connected, and as different materials and objects have different reactions to lighting, they look more realistic than before.

With the previous probe-based global illumination, small objects and details didn’t have any effect on the global illumination. However, thanks to the new system which works on a per pixel basis, all the objects in the world and every small detail can have an influence on the lighting.

Quentin points at a controller on the table we are sitting at.

“For example, if you take the controller here as an example, when we had probe-based illumination, you wouldn’t be able to cast those small, detailed shadows. But now, we can catch these small indirect shadows on their surroundings – the controller casting a shadow on the table, with the TV screen shining a light on the scene for example. It’s much more precise, and much more realistic – and more physically accurate as well since it reflects rays of light in real life.”

It’s much more precise, and much more realistic.

This also connects to the weather system on Pandora. For example, when it’s foggy, some light will be obscured, and the fog itself uses Ray Tracing to indirect light, creating a systemic interaction.

“The systems kind of feedback on themselves. If the sky is completely clouded, you get less direct light from the sun, making the environment darker. It’s all connected,” says Oleksandr, and adds that one of the challenges they have faced in the past is light shining into interiors, which has been a difficult problem to solve.

“We have worked a lot on this and been pushing the boundaries of the tech to make it work – which it now does,” he says with a smile. “And what’s even nicer is that since we don’t rely on baked data, we can actually support opening doors into dark interiors which will become bright due to the light shining in from the outside. It sounds like a basic thing, but it’s something we’ve worked on quite a while, and it’s nice to finally achieve it.”

Speaking of bake times, Ray Tracing has been pivotal in shorten iteration time for the artists working on the game, as it eliminates long bake times and gives artists instant feedback and removes some of the problems with baking. For example, to get the impression that you would get in the full game, the team had to bake things on a build machine, which would take a lot of time.

“With Ray Tracing, if an artist moves an object, it will instantly update the lighting. This means that artists can create windows and objects and instantly see what it will look like in-game for the players,” explains Quentin.

By working closely with the artists on the project, the Snowdrop team has been able to make the tech better and better with direct feedback, considering how artists work with Ray Tracing.

“It’s really a collaborative effort,” says Oleksandr. “When we started the processes in the development of the tech, it showed promise quickly. But once we committed to it and started using it to produce Avatar: Frontiers of Pandora, we had to tackle a lot of challenges, and that’s where our collaboration with the artists really started.”

We had to tackle a lot of challenges, and that’s where our collaboration with the artists really started.

By working together, the artists worked to understand how to implement Ray Tracing in their work, and the Snowdrop developers figured out the best ways to optimize the tech. The collaboration has yielded great results, and the instant feedback from artists to the Snowdrop team has proven invaluable.

“When we work on the tech side, we work with simple scenes with a red cube that you can see the light bounce off from,” explains Oleksandr. “However, once you go to a real level, you can see a lot more, and realize that the artists see issues that we don’t necessarily see since it’s so much more complex.”

Quentin adds:

“The feedback has been great. For us in the Snowdrop team, what we do is physically based, and sometimes you try to be as realistic as possible, but it can actually be too realistic for artists, since it hampers the player’s experience,” says Quentin.

As an example, Quentin brings up light during nighttime in the game. During the night, it will (quite obviously) not be much light in the environment, but for the player, it’s too dark and takes away from the gameplay.

“If the player can’t see anything, it will take away from the experience, so sometimes it might be that they need to boost light in a certain area to make the player experience as good as possible. It might not be the most realistic lighting, but it’s necessary to make the game fun.”

Moving forward, both Oleksandr and Quentin see many opportunities with Ray Tracing and how it can be further evolved.

“What we are doing right now is basically tracing rays from the surface of objects into random directions. If we hit something, we hit something. But in that sense, it’s limited since the light won’t bounce indefinitely. What I think will be our next technological step for our future titles is “Path Tracing”: shooting rays which bounce and bounce and bounce. At the moment we are still working on solving some challenges, but it looks promising and it’s definitely something that we want to keep pushing,” says Quentin.

Right now, we are at the tip of the iceberg.

The Snowdrop team’s goal is to make all new technologies developed on Snowdrop available for any team at Ubisoft that is using the engine to develop their games. Ray Tracing is no different and is now available for anyone to leverage.

“There are things we are working on to get better precision and quality,” says Oleksandr. “Right now, we are at the tip of the iceberg, and there are a lot of exciting and interesting things that we want – and hope – to do in the future when it comes to Ray Tracing and lighting in games.”

More from Snowdrop

• An Evolving Pandora, Snowdrop’s dynamic world
• Crafting Pandora’s Breathtaking Landscape, Snowdrop’s scattering system

Avatar: Frontiers of Pandora © 2023 20th Century Studios. Game Software excluding 20th Century Studios elements: © 2023 Ubisoft Entertainment. All Rights Reserved. Avatar: Frontiers of Pandora and the 20th Century Studios logo are trademarks of 20th Century Studios. Licensed to Ubisoft Entertainment by 20th Century Studios. Ubisoft and the Ubisoft logo are registered or unregistered trademarks of Ubisoft Entertainment in the U.S. and/or other countries.

“In Avatar: Frontiers of Pandora, the player can directly do things that affect the state of Pandora,” Kunal Luthra, Lead Technical Artist, says. “You defeat a big facility belonging to the RDA, the main bad guys of the game, that’s polluting the world around it, it will become abandoned and overgrown – nature will be reclaiming the area. The forest will recover from the pollution, animals will return. And as you push back the RDA across Pandora, you will see it directly reflected in the world around you.”

“The player can tell straight away when they are approaching a Resources Development Administration – the RDA – facility,” Kunal says. “The plants are dying around you; the wildlife is dying around you. Harvest materials are destroyed, and suddenly you can’t harvest anything. The pollution takes away gameplay opportunities, and the only way to get them back is to destroy the facility. And as soon as you’ve done that, you can see the pollution start to disappear in front of your eyes.”

The system was originally built for Tom Clancy’s The Division 2, where civilian settlements get upgrades as the player helps them recover. It has then been expanded and built upon for Avatar: Frontiers of Pandora.

We can change almost anything. We can change the geometry of the level area, we can load and unload entities we no longer want.”

“We can change almost anything. We can change the geometry of the level area, we can load and unload entities we no longer want,” Daniel Edenbrandt, Senior Gameplay Programmer, says. “We can have new geometry for the animals and NPCs to traverse. It also affects the shaders, that decide how objects and environment look, so even when we don’t change anything gameplay-wise or geometry-wise in a wider area, we can change how we render the plants and environment using what’s called vertex shaders and pixel shaders.”

The system is also built this way to empower the game’s level designers, as it gives them the opportunity to create bases with engaging gameplay together with detailed environment.

“We have scripts in the Snowdrop editor to help the designers generate a variety of foliage depending on what area of the map you end up placing your base,” Kunal says. “You can design it once, and then the system spawns the correct plants automatically based on that. This means that we can create the most immersive world for our players, where the lush and vibrant vegetation populates the entirety of the vast landscapes of the Western Frontiers of Pandora.

We show and hide whatever is appropriate, with bespoke geometry. It’s like layers in Photoshop, essentially.”

“When you design a facility, you place the props and everything that should always exist, then you place the props that should only exist when it’s operational and then props for when it’s abandoned,” Daniel says. “Then we show and hide whatever is appropriate, with bespoke geometry. It’s like layers in Photoshop, essentially.”

Pollution is a whole system in itself, that directly interacts with the dynamic world system.

“The way it’s set up, any gameplay system can access the pollution, and it is also available to graphics shaders running on the graphics card using a pollution texture,” Daniel says. “We can have any range of pollution and if we wanted to, we could define the amount of deterioration on any material or gameplay object – like a harvestable, for example – using these shaders, instead of having multiple versions of the same object just to showcase the amount of pollution.”

The goal of the dynamic world system was to tie everything together and to make every landmark unique. It’s meant to give the player an incentive to go back to previous areas to reexplore places they’ve already visited and to create a sense of the passage of time. But it’s also a part of the game’s message.

“The basic purpose of the dynamic world is to show how bad human actions can be for the environment, but that we can take it back and help it recover,” Kunal says. “The facilities will still be there, but as you progress, they will be taken back by nature.”

More from Snowdrop

• Shining a Light on Pandora, Snowdrop’s Ray Tracing
• Crafting Pandora’s Breathtaking Landscape, Snowdrop’s scattering system

Avatar: Frontiers of Pandora © 2023 20th Century Studios. Game Software excluding 20th Century Studios elements: © 2023 Ubisoft Entertainment. All Rights Reserved. Avatar: Frontiers of Pandora and the 20th Century Studios logo are trademarks of 20th Century Studios. Licensed to Ubisoft Entertainment by 20th Century Studios. Ubisoft and the Ubisoft logo are registered or unregistered trademarks of Ubisoft Entertainment in the U.S. and/or other countries.

Kunal Luthra, Lead Technical Artist, David Österlind, Lead Environment Artist, and Carl Leonardsson, Senior Programmer, are three of the people that have worked on the scattering tool to help the Avatar team build an immersive Pandora.

“Simply put, the scattering system is a procedural placement tool,” David says. “It uses a node system that connects objects together in a parent-child relationship based on the radius around the objects. Maybe a tree has certain plants that grow around it, or a river has a certain type of pebbles and rocks on its banks, which in turns have certain flowers that grow around them. Once we have set up the logic we can populate the world in an instant, allowing for quick and easy iteration. It’s advanced, but the core idea is very simplistic, in the sense that everyone can grasp it.”

“We consider scattering to be the first pass,” Kunal adds. “But it’s a good first pass! When we’ve locked the scatter, we can go in and massage the content, add, remove, and move objects – adding that artist flair. And if you have an area that is extra important, where you know something important will happen in the game, artists can go in and handcraft those specific areas without using the scattering system first.”

Procedural world building tools are not new, games like Far Cry 5 have used them for a long time, but the scattering system in the Snowdrop engine was built from the ground up based on the specific needs of the project which brought a range of new innovations. After all, each procedural generation tool does things differently, based on the desired outcome. But with this level of fidelity and detail, it’s easy for the tools to bloat and become hard to use.

For artists, this tool is invaluable.”

“The scattering interface in Snowdrop is meant to be as art friendly as possible,” Kunal says when asked what improvements the team has made compared to other similar tools. “The system is not just driven by a code recipe to generate some procedural, arbitrary result. An artist can, from scratch, fill out a map based on rules, with specific assets, the child-parent relationship between objects, hierarchies, and build a nice, believable world in a user-friendly way. For artists, this tool is invaluable.”

Originally, the scatter system might have been built for Avatar: Frontiers of Pandora, but by now it has become a general Snowdrop feature that other games can benefit from.

“We did cater it for this project,” Kunal says, “to create the Pandora we wanted, but we always keep in mind how to keep it scalable and streamlined for other Snowdrop projects in the future. The more projects use it, the better and more comprehensive and advanced and easier to use it becomes.”

“It is definitely integrated into Snowdrop overall now,” Carl says. “That’s also something that we want to work more on, to make it better and easier for other projects to apply.”

Once an area has been automatically populated, it’s easy – and fast – for an artist to go in and tweak and fix things to their specifications.

“And we can scatter in layers,” Carl says. “We can place objects in layers. You might have your aquatic systems – you can have scattering at the bottom of a river, at the surface, on the cliffs next to the river, perhaps on the floating island just above it.”

It’s also important for the team behind the scattering system to be in contact with other teams, like game design, to make sure that the areas it produces fit the rest of the game.

“It might be artist driven, but it needs to go hand in hand with level design and game design, for example,” David says. “For example, interactive plants might need to have a certain distribution, or crafting materials can only be found on a certain tree with a certain type of bark. That needs to be considered as well.”

We can scatter on other assets, on props, anything really that we think should have it. That’s very powerful.”

The system is also built to be fast.

“You can have your graphs that describe the rules on one side, and the game on the other,” Kunal says. “We make an adjustment and in seconds we see the results. That’s another of its key strengths, that the iteration times are so fast – you can iterate, and iterate, and iterate quickly, and not have to wait for ages until your changes are reflected in the game.”

“When we started, we were only placing things procedurally on terrain,” Kunal says. “But now we can scatter on other assets, on props, anything really that we think should have it. That’s very powerful.”

“At the end of the day, it’s about keeping the player immersed, consistently, through the whole game. And if there are some pockets, anomalies, that we can tweak, then we do that.”

More from Snowdrop

• An Evolving Pandora, Snowdrops dynamic world
• Shining a Light on Pandora, Snowdrop’s Ray Tracing

Avatar: Frontiers of Pandora © 2023 20th Century Studios. Game Software excluding 20th Century Studios elements: © 2023 Ubisoft Entertainment. All Rights Reserved. Avatar: Frontiers of Pandora and the 20th Century Studios logo are trademarks of 20th Century Studios. Licensed to Ubisoft Entertainment by 20th Century Studios. Ubisoft and the Ubisoft logo are registered or unregistered trademarks of Ubisoft Entertainment in the U.S. and/or other countries.