Research Project Summer 2021

Research Project Summer 2021

For my research project I engaged several different software platforms throughout the summer. I was able to obtain 3Ds Max and begin learning the basics of modeling techniques in the program which I will use in my upcoming internship with the Institute for Simulation & Training who are interested in having my work on the 3D assets as well as Audio assets and blueprints that I will need to research and learn how to create.

As capstone projects have come to a close, I affirm to report that I have also learned more about audio implementation in engine. I was responsible for creating and designing SFX for in game play, VFX and music cue functions, as well as produced an entire soundtrack for the entire game. I worked with the different team members, including programmers, animators, and team leads as well as the musicians to guide and direct the overall sound and audio for the game. This required research in different ways to understand how these sounds would be implemented in engine and help create our final product. When I joined the team, they were at a 5-10% completion in terms of audio, and my arrival brought the project up to speed and completed 100% of the audio requirements, with time to spare for iterations. This was a massive achievement this summer for our capstone team.

For additional audio research I also learned how to use Audacity to begin creating an audio portfolio and a website to showcase my abilities. Research has shown that while Audacity is a fairly simple program built for people with limited audio experience, my new goals will include recommendations from Nick Zuccarello to learn the new UE5 system of how to balance audio in engine with a more holistic approach, and my previous audio experience with different Digital Audio Workstations, sound and audio editing software will only help me advance further as I begin my internship, guiding me towards as successful career

Common Art Sprint #5

 Common Art Sprint #5

For this week I made certain all of my 3D assets were up to date with any finishing texture modifications in the new lighting that has been developed to make sure everything looks copasetic. I noticed that for some reason the nightstand texture was not working, so I discussed with Desiree and we decided to unlink it from its drawer and then I was able to get the texture updated. Overall I am pleased with how the assets I worked on look in the space, especially the lamp because I feel like it is one of the focal points in the room and really illuminates the space and creates a realistic VR experience:

For audio I worked extensively with Manny to get all of the levels correct and make sure everything was working like a well-oiled machine. The main difficulty was the Anton Threat track which has Anton's voice. I think this track could still be boosted in  ways, but the problem with the technology is that the Oculus woofers have no bass - there is not a very good way to get lower resonating sound frequencies, it is almost like using a pair of ear buds. It might be beneficial to write the company and do research to figure out why that is the case. I will say the speakers that are in the current device are my biggest hang-up with creating immersive experiences in VR right now. Sound is at least half or maybe 75% of the experience, and without better equipment, modifying the wavs to fit the device will help, but it will only take it so far.

Overall I think our virtual reality world has great sound and feels immersive, there are a few minor adjustments that could be made such as attenuating or localizing the Front Desk sound to the receiver but myself or Manny will need to do research into how to program certain details like that.

3D Art Portfolio #3 - Integrated Audio Scene: Week 2

Switching gears this week after doing more research and connecting with Matt Tracy my focus became generating an audio piece against a pre-existing cutscene. I choose a night raid scene from the video game Battlefield because it has the first person shooter drop into an already immersive war scenario. I edited the scene visuals down to just under one minute because that is about how much time you have for someone in the industry to hear what you are doing and know that you have skills.

Edited scene:

I also picked the scene because it has a nice beginning, middle and end with the initial parachute descent, the hectic battle as you make your way through the scene, and then a closing with the tank smashing through a wall and firing as the climax to the scene. First I had to gather all kinds of sounds in order to simulate the experience and this is the most audio I've ever collected for a project:

I approached the soundscape similar to a 3D art approach by thinking of aural experience as having a foreground, midground and background, so there will be layers to what the audience is hearing. In my first pass I needed to incorporate all of the first person shooter weapon sounds the firing of the automatic and the reloading sounds, and also things like when the person first lands on the ground, having sound effects to give that sense of drop and hitting with their body.

By far the most challenging aspect in this initial first pass was to get each shot fired to match the kick back of the gun, and I had line up each of these shots going frame by frame slowly, playing back over and over to get it right. This was by far what I've logged the most hours on, because if one shot was off, in that split second the human eye can see and hear something isn't right. It became very specific. I also included a couple sounds in the intro and in the outro I wanted to get the tank smashing through the wall because to me this was also a main event that should be up close and visceral.

First pass video:

After I had things sounding decent for the first person foreground, I approached adding the other guns shots occurring from surrounding comrades which proved challenging almost like finding the correct rhythms against what the first person shooter was doing, and those sounds had to be heard in the midground but not compete with what I had already created. I also began adding bomb explosions both close and distant, to get the right feel for the auditory experience given where the bomb was being dropped and how much the impact would affect the viewer.

In the distant background I needed the air raid sirens which gave the aural tapestry a nice backdrop as sounds played out. Finally during this pass I tried to hone in on certain sounds that were missing from the original cutscene reference to improve the overall sound re-design such as the power line boxes that explode when they get shot, two different bullet attacks to the first person one of which injures the character. Overall the soundscape is now incredibly visceral. The next step will be to add voices from the soldiers as they yell and call out to each other during the battle.

Second pass video:

3D Art Portfolio #3 - Integrated Audio Scene: Week 1

3D Art Portfolio #3 - Integrated Audio Scene: Week 1

For this week I wanted to complete the level I plan to integrate audio into, and using a factory asset pack I was able to restructure several of the areas into a closed set of rooms to create a sort of factory corridor that I can then set the sequencer up to move through and create audio scene I am going for. Here are several images of the level build:

In the above image and its sequence the sounds I will focus will be the forklift as you pass by. In the next will be the sounds of the conveyer belt and the robot welding arms movement and welding sounds, and also noises of workers in the background moving equipment.

As the sequencer moves through the next part I want the systems and the conveyer belt to be heard and in close proximity as the camera moves above it.

There will be more robotic arm sounds and I also want to create certain electrical hums in places.

This next corridor could have certain saw type sounds that are behind the wall, and if I can find other machinery assets to add these next two or three areas could have different sounds also.

The above image shows where I can get some light bulb electrical type noises happening, and focus on how the audio would change going through a type of room like this. Then in the last image the last big open room, I would like to incorporate some crane type assets and sounds.

The next weeks phase will be gathering the audio files generating the list I first created so that I have all the audio assets I might need imported into the game engine.

3D Art Portfolio #3 - Integrated Audio Scene

 3D Art Portfolio #3 - Integrated Audio Scene

For my final portfolio project during the summer, I decided since I would like to go into military simulation and focus heavily on Audio production, I wanted to create a type of simulation room in a first person shooter style that would be for training purposes, and also incorporate a gaming feel - an integrated audio scene. In order to focus primarily on the integration of audio, I will be using existing 3D assets I have created, as well as looking for asset packs to fill out the space, so that it could be used as a VR simulation room. Once I begin work and understand the scope better, the space could become larger, but for now the goal will be to contain the scope in a smaller space.

My 4 week schedule is as follows:

WEEK ONE: create and design room concept and make a list of existing assets and/or using an asset pack and also generate a list of audio elements that will be needed for the space, and begin level build out with initial lighting pass setup.

WEEK TWO: finalize room build out and begin work on gathering audio elements and begin creating audio elements for the space and getting the assets into engine to test out levels immediately, begin work on soundtrack music to go along with the space in this initial stage so that the audio elements work together as I build the music compositional backdrop.

WEEK THREE: Finish music structure that can loop during the experience and test and finalize levels of SFX and any additional audio elements. Final pass on 3D objects and asset layout. Second lighting pass and test all integrated elements.

WEEK FOUR: Final music mastered and implemented. Final lighting pass and adjust any textures if need be. Test experience with players and get feedback on audio elements, check with Rick, Chris and Nick if they have time to check it out and offer feedback.

Environment Concept: Factory hallway or corridor / military type hallway with objects and machinery

For the space concept I'd like to work with the idea of the player moving down a long hallway and passing by different types of machinery. This will be a heavy opportunity to work with attenuation and sound levels, so that when the player is passing by different machinery or sounds they might hear, such as workers talking loudly or shouting at each other, the player will have more of an integrated experience. In terms of the music composition, I would like to generate a track that runs the distance of the hallway, with the possibility of the music loop changing as the player gets to different check points. This will be the first goal of the space, and to have different sounds occur down the space, but this way I can work with the idea that the player is only really traveling forward to keep the scope more limited.

I will use a factory asset pack to build out the space quickly and then generate some lighting, so I will immediately have a working level without having to full design a space, and can work directly with integrated sounds on a linear passage way, which also gives me an opportunity to focus on stereo sound, such as what the player is hearing on their left versus their right ear, and develop a three dimensional approach to stereo sound in the experience.

I want the hallway to have the feel of a music video film production shoot, so the goal will be to see if I can get the kind of drama I want from the experience while still creating realistic soundscapes to go along with the music.

Here are some reference examples of the type of hallway I will create with the asset pack:

Audio Assets: preliminary list

Machine types of sounds:

1. Metal saw

2. Grinder

3. Welding

4. Hammering

3. Oil Pistons

4. Electrical charge

5. Intercom

6. Water Hydraulics

7. Gears

8. Forklift hidden

9. Forklift visible

10. Conveyer belt

11. Workers talking / shouting

12. Ventilation

13. Machine drone

14. Machine arm

15. Machine shutdown

Music ideas: industrial electronic - 4 different sections (solitude, trepidation, intensity, havoc)

3D Art Glock Textures

 3D Art Glock Textures

For this week I was trying to get the Glock into Substance so I could do what texturing I was able to which wasn't as much as I wanted. Right now I have basic textures on it and there is definitely more work to be done to give it a more authentic look. I am hoping that I make another push to finish it as I move on to the 3rd portfolio piece, or continue work on it during venture track. Overall right now it has the base textures I will use, but I still need to add more details such as the grip and the lettering, as well as get the surfaces to look more natural.

Common Art - Lighting Corpse Bride

 Common Art - Lighting Corpse Bride

For our lighting project we were given the interior scene from the Corpse Bride VR project and given free reign to creating lighting as we saw fit. I really liked the glowing dust effect the team already had in the room, so I decided to keep only that element.

In terms of adding light I started from scratch, added a dim skylight, one directional light, one spotlight over Emily at the piano, exponential height fog, post processing volume, sphere capture and then used point lights to create the look I wanted:

I wanted to draw the viewers eye to Emily, so I kept most of the lighting soft but strong enough to create some shadows and depth to the screen shot I wanted to create. I was able to use low point lights over the table tops to create some light and break up the darkness in places. I wanted kind of a purple red above as if maybe there is a fireplace up on that level.

I like how I achieved sort of a silhouette effect around Emily since she is the main event in the scene. I also used some orange lights in the next images to create something like torchlight on the walls, and I think if I were going to add anything to this scene it would be just a few torches hanging on the walls like a type of sconce just to have a place for where the light is coming from, or maybe a large chandelier - even so, I think the room design is beautiful and just needs an object or two to give a reason why the light is there:

I also added a rectangular light to try and draw out the bone design on the front of the bar because I thought those details looked really cool and just needed some highlight to bring them out.