NMFO: First and foremost, why don't you tell us a little about yourself?
AE: I'm a visual FX supervisor. I'm the department head for visual FX on set. That's the best way to consider how it relates to the on-set work. I also follow the shots all the way through from the production side on-set and help with the planning on set through their completion in post-production.
We acquire the stuff on-set, I get the extra pieces that I need on-set for visual FX work, and then I follow those shots all the way through the life cycle until we deliver them back to the editorial department at the end, and then cut them back into the film.
NMFO: What is a VFX artist?
AE: Visual FX are shots that are created in post-production. So, a lot of times, people conflate special FX and visual FX, and part of that has to do with their name of them.
They're called special visual FX. They used to be called special FX sometimes, and sometimes there's a lot of overlap. So, special FX are FX that are created on-set for the camera. For example, flipping a car, blowing up a car, lighting things on fire, making smoke. Those are things that they do in real life on set. But then, some kinds of FX are a little bit too dangerous, or just physically impossible, to do on set. For example, creating dinosaurs or making digital doubles, like doing stunts that are too dangerous for humans or making large environments that are possibly too expensive to build.
We would do that in visual FX in post-production. These days, that means mostly computer graphics, but we use a lot of photographically sourced material as well. We might be mixing miniatures in there. We might be mixing other photographically sourced images in there, or we might be painting them up from scratch, not just generating them with computers, like 3D computer graphics. We might essentially do Photoshop-like old-style matte paintings and then compositing those into the shots.
So that's visual FX versus special effects.
NMFO: I was even thinking, too, I was watching a movie the other day, and now, in movies, everybody has their phone screen. or a computer screen. How those end up getting composed into the film.
AE: Screens are another case that comes up a lot because oftentimes, especially now with all the stuff like people talking in video chats, sometimes the other side of the conversation hasn't been shot yet. So, what would they shoot on set even if they could shoot? They have nothing. So, they'll shoot with a blank screen or with a green stand-in or something, then later, once the stuff's been recorded, we'll composite the screen in.
In the past, they would even do that optically because it was, depending on the situation, sometimes impossible to photograph the screen because of mismatched frame rates or because the quality wouldn't have been good enough. So, they would still, even before we were doing digital compositing, when it was still optical compositing, they would use film techniques to composite stuff onto screens. Putting screens in on post has been a longstanding tradition in this industry.
Also, for continuity's sake, sometimes it's impossible to have the screen thing coincide with what's supposed to happen in the scene.
NMFO: What made you want to get into video and visual FX?
AE: Well, I wanted to do computer graphics from a young age. When I was about 10 years old, I knew that I wanted to do computer graphics. I saw stuff on TV. Tron was an influential film for me in terms of just being exposed to the concept of computer graphics. I saw Tron, and I thought, “Boy, that's really cool.” In all the making of videos, they talked about how computer graphics were the recognizers, and the light cycles that were flying around were achieved with computer graphics.
At the time, as a child, it seemed inaccessible because you needed a supercomputer of that era to do the work at all. But, if you fast forward to where I was in my late teens and early twenties, suddenly, you can start to do this stuff on personal computers at home.
I was able to start experimenting with computer graphics and complete that dream as a child of making stuff from an idea and being able to use a computer to make it. That was when the entire industry shifted towards nonlinear editing using digital workstations for manipulating sound, and Photoshop was still relatively new at the time. So, I had a realization that this was going to become the universal media manipulation tool for pretty much every format of media that you could imagine. Also, based on my interest in computer graphics, I realized that was a power tool, like a Swiss Army, for making images. That really turned me on and made me twice as excited about mastering those tools.
That evolved into my career in visual effects because it's like, “Well, what are you going to use all these computer graphics? What are you going to use all these pictures for?”
Then I started learning more about classic visual FX techniques, process photography, optical printing, shooting miniatures, the classic glass mats, and all the classic ways of doing visual FX and post-production that keyed into the computer graphics.
Those techniques are all still relevant. We do the digital versions of the classic techniques. Then we have new versions that would never have been possible using the classic techniques, like the way that we can essentially hand off or take over the entire frame using computer graphics in 3D. That type of thing was impossible until computer graphics became one of the tools.
NMFO: So, it has been a good organic progression of learning more, seeing how the technology progresses, and fine-tuning your skills. Obviously, you're passionate about it.
AE: Oh yeah, there's always something new to learn. That's one of the fun things about visual FX. If you're not a person who enjoys constantly upgrading your skill set, then it probably wouldn't be a career for you.
NMFO: It's not a place to stay stagnant.
AE: No. Especially now, there's all this new AI stuff that has brought a new set of tools to things. I think a lot of people are concerned about the capabilities that AI brings. In a lot of ways, it removes more of the scut work. It allows us to focus on more of the high-level stuff rather than much of the busy work. We can use a computer like a bot to take care of a lot of stuff that used to be tedious manual labor before. Now, we can have the AI do it and get results that we would normally have to toil for hours to produce. Like rotoscoping, which is cutting mats manually, literally by tracing them. The AIs will be able to do that more efficiently. Then we could just focus on the creative work and making it look awesome and creating the exciting images everybody's used to seeing, not focused on all this minutia that takes a lot of time.
NMFO: Work smarter, not harder.
AE: Yeah.
NMFO: Because you've been in the industry for so long, how do you see the trajectory going as technology advances? You've touched on it a lot here with AI and how it can be beneficial. Everybody talks about how it is going to replace jobs. But it could allow us to sharpen our skill set and do things that are more advanced than really paying attention to the things that just take time.
AE: We've had very high-level capabilities. I remember when 3D computer graphics type techniques started to mature, and we were doing things like face replacements and de-aging people and digital doubles. You may have heard this phrase: the uncanny valley.
So, it has to do with human faces, characters, and photorealism. The more you approach reality in terms of a simulation or trying to create something that looks like reality, the 80-20 rule. The closer you get to 100%, the further away you get. That last 1% or 5% or 10%, it gets more and more difficult to achieve because the tiny inaccuracies of the human eye are just very sensitive, especially to human faces.
Anything wrong with the motion, the way the face animates or deforms, subtle things like eye gaze, the lighting. If anything looks the slightest bit wrong, it's rejected as looking fake. We're very sensitive to that. So, that's the uncanny valley. You must go down into the valley before climbing back up to the other side of realism. That's been a place where we've been stuck in a lot of ways, especially when it comes to replicating perfect humans.
People have been stressing about digital characters or replacing humans for maybe 15 years. We're barely getting to the point where we're starting to nip at the heels of the uncanny valley effect.
Fifteen years ago, there was some good stuff that was being done. We’re talking about the most elite studios spending huge amounts of effort on individual shots to achieve stuff that was still a little creepy and not perfect. Now, we're finally getting to the point where we can create a higher volume of shots at that same not-perfect level.
We still haven't even achieved the perfection part. So that's one of the places where AI might help a little bit because it has some of the tools to help with fine-tuning. But it's never going to completely replace human performance because human performance is what drives it. I mean,
Let’s take a film like Avatar, Way of Water, as an example. Everything is done as performance capture before the CG is, then essentially, driven by the performance capture done by actors. The part that a lot of people leave out, even people who do this, because sometimes it benefits them to omit the gory details of how it's all done, is that hundreds and hundreds of man-hours of effort go into massaging that mocap performance into something that can be applied to the CG characters successfully. The CG characters that need to be produced need to be fine-tuned so that it doesn't, again, enter that creepy uncanny valley and lose all its appeal, and the audience rejects it because of the creep factor. So, if anything, it's in some ways, the demand for this level of work has increased, at least for the time being, the amount of human effort necessary to complete the shots.
Especially if you look at the budget of a film like Avatar, they're not just burning through money. They're spending it on humans to complete these shots. So, I think, in some ways, those concerns are overblown. They're not focused on the right things.
There are some things I do worry about in terms of the junior-level folks who would, oftentimes, the way you pay your dues is by doing a lot of the more labor-intensive stuff, and if there isn't a pipeline in it, how are they going to learn?
NMFO: How are they going to learn the fundamentals of how to get the job done if there's an AI already doing it for them?
AE: Yeah, I worry about that. But it also means they're all starting from a higher launch platform. They'll have access to all these tools. So, they'll be able to produce very high-level stuff more easily. It's going to be more constrained by their imagination than it is by their skill set.
Also, the amount of labor that they could put into stuff. They can get through a lot of that same scut work themselves and produce cool stuff. Even now, if a student is very creative in terms of how they apply their efforts and the projects they approach, they can produce stuff at a level that, even two years ago, they couldn't have imagined because of the quality of the tools, the level of automation that's available, and the speed in general. Increasing the amount of capacity that they have with their computer.
The software has become much more sophisticated. The last juniors I hired that were doing CG, the stuff they were doing was very high-level, given the size of the team. It was a very small team. They created a whole CG animated short with just two people. It was very impressive.
That's part of the reason they got a job, too. It's because they create very impressive work.
NMFO: You know you've worked on huge projects, some Marvel stuff, including Iron Man 3 and the Amazing Spider-Man, and some local New Mexico productions, including The Cleaning Lady and Vengeance. What are some of your favorite projects that you've worked on?
AE: Let's see, I've been very fortunate in my career to work in the animation part of the business. I never worked at Pixar, but I worked at DreamWorks. So, full CG animated film stuff. I've also been able to work in the visual FX part of the industry. I have that experience of making a film, just whole cloth from scratch. That’s where the process is like live-action stuff, especially in TV, where they have the writer’s room, they workshop stuff a lot, and keep working on it till they're happy with the story. That's very much the animation process.
I've been able to watch many films come to life through that process, seeing the early screenings that were oftentimes no animation at all. It's just storyboards. They cut them together into a reel. You see a moving storyboard version of the film, then you see it slowly come to life sequence by sequence as they decide, “Okay, we're going to lock this sequence, and we're going to put this in production,” and start animating it.
Then, it gets replaced with the fully animated and rendered stuff. It's hard to say what was my favorite part of the industry. I really liked the structure of how animation is done because it's so thoughtfully approached and so carefully executed. I mean, you could say the level of planning is nothing like the way live action is.
Live action's a lot more like a scattershot where they have a script, but you shoot all this coverage of stuff that is at least 75% of it, 80% of it's never going to make it into the edit because of how you're doing the coverage. It's just going to get thrown away. If you have a higher budget, probably 90% of it, if you're shooting high ratios.
So, it's interesting to compare to animation, where you know the shot you're making before you make it, compared to live action, where you know the scene you're making, but you don't know if it’s going to make it into the cut until it gets edited.
My favorite animated project that I worked on was Kung Fu Panda. My favorite visual FX shot was a film that I worked on called G-Force, mostly because it was just a fun project to work on. They had these talking guinea pigs, and the show was good. A former visual tech supervisor directed it. So, it seemed like they knew which shots they wanted. The shots seemed to be well-planned, and the characters were fun. They were fun to work on.
You can see there's a theme there. They're both cartoony characters. I like that character-driven stuff. When it comes to visual FX, a lot of the projects I work on have this in common: I like doing set extensions and matte painting-type stuff. Things that are more classic visual FX approaches. Most projects I've worked on have had some level of that. There's always a little bit of something in every single one for me.
NMFO: I want to go back to the beginning of our conversation, where we really tried to talk about the life cycle and the pipeline of everything that it is that you do. Can we go back to that and touch on what the structure of your role does throughout the entire production?
AE: Let's talk about visual FX. There's always a script, and the first thing I'll do, even before I go to set, I will break that script down with a producer. That just means we go through everything. We read the entire script. Every single scene. We try to imagine what the shots might be, and it is difficult to translate a script into actual individual visual effect shots. That's the first stage of beating it, breaking it down, and it starts the conversation with filmmakers.
Typically, we've made a shot list for them. Sometimes, we do it in that first pass. Sometimes, they also have a shot list that they give us, and then we create our breakdown based on that. That starts the conversation and allows us to start hashing out the details of what is going to go in, what's going to fill the screen for each one of these moments that we break down from the script. It also helps us determine what resources will be necessary to complete that shot.
We create that bid, and we deliver it to production. Sometimes, there's a bit of back and forth over that, over which shots will be in there, what they want to do, and what they don't want to do. So that creates the framework of the shots that we know we need to get on the day when we show up on set.
After that happens, usually this happens about the same time that they're doing their prep or in the early stage of the pre-production process. Once they're ready to go into prep and pre-production, I'll attend those prep meetings and sit down as we do a full read-through of the script. Sometimes, we'll do it scene by scene. For episodes, we'll do that per episode. On features, we'll do the whole thing at once, but there are multiple meetings. Oftentimes, we'll get pulled into meetings, especially if there's an overlap with other departments where visual FX will be part of the process. We'll have meetings with almost every other department head because if it comes to things like, let's take an example of just the set design. There could be partially visual FX and partially physically built sets. We might be following the design of the set designer, and the only reason they didn't build it physically is because of the space that they had. The volume that they had to build the set in just wasn't big enough. We extend their design into a visual FX version of it. We don't want to mess up their design concept. So, there's a lot of back and forth between the set designers and visual FX. Those are great collaborations. I love those.
We might also work with wardrobe because we might have characters with digital prosthetics for various reasons. They might have parts of their wardrobe that are physically impossible to do. A good example might be Furiosa's little claw in Fury Road. It was partially a physical thing, but they had to remove parts of her body to complete the gag. So, we'll have those conversations with the department heads so they know what we need from them, and we're all on the same page about the expectations of who will do what. That makes sure that the visual FX can be executed in the most efficient way possible and so that there's not a disconnect between what we're acquiring on set and what we're going to be doing later in post.
We collaborate a lot with special effects. There are always combo platters with special effects where they'll flip stuff, blow stuff up, or even stunts. Sometimes, wire work, we'll have a lot of wires we'll have to remove, and the way that we can weigh in on the way those rigs work.
Once everyone is on the same page and we know what we need to do on the day, then we go into shooting. Depending on the work that needs to be done in post and visual FX, I might need additional stuff from set on the day the main camera crew is not. Examples of that would be if we need to do 3D stuff that involves the set or even just a 3D creature, that sort of thing. We typically need an enhanced reference level from a set that normal cameras can't record. That would be things like physical surveys of the set itself. We must survey the sets as built to have the precise scale of the environment. Sometimes, we'll do actual scans.
There are other pieces of reference that we might need to capture, like photographic reference of props and people in wardrobe and set pieces. Then, there's a specialized reference we get, which we can use later for image-based lighting. Especially if there is a CG character in a shot, we'll always want to get that because that will help us model the lighting in the environment as it was on the day when that scene was shot. There must be a visual FX representative, or two or three, depending on how big the set is and how many cameras are running. You might have to have multiple.
Then, we log all the additional camera data from the main cameras. That's helpful in post if you have a visual FX representative on set because things almost always change. It's a very fast-paced environment. Suddenly, they're zigging instead of zagging, even though we had a plan. The visual FX person on set can make sure that the plan is updated correctly, taking into consideration visual effects concerns down the line.
Now we have got through all the acquisitions, we've got all the piles of footage and all the additional references. Then, it all goes to editorial. We stand by for a bit while they figure out what shots from the coverage are in the film.
Once the editorial is done, then we know which shots and takes are going to be in the film. They then provide us with a list and the footage, and we know what to work on. Visual FX is labor intensive, so there's no reason to execute visual FX that will get cut. That would be pointless.
We work exactly to the cut length. Plus, we add some frames to the head and the tail just in case they decide to slip the edit in post. It’s just enough to create a little safety without increasing our effort to deliver the shot. Otherwise, we just deliver exactly the cut length, which would reduce the flexibility if they needed to slip the cut just a little bit.
They turn over all that footage, and then we get to work. We bring all the references that we got on set. For example, a 3D set extension is a medium complexity case; we have that set survey that measured the set exactly. We know where the actual physical camera was when it was shooting it. Now, we need to extend it to have more stuff behind it. We would produce all the assets necessary to do that. If it was a large 3D environment, like a futuristic city on a hill, we're flying over it. We must create all that stuff in 3D. Then we must match what's called match move the camera. You are creating a virtual version of the camera in a virtual space, in the same place as the real camera was relative to the real set.
NMFO: What is the timeframe from pre-production to when you're done with it back to editors? What does that time frame look like for features versus TV? And what does that time frame look like when you don't have somebody who's there on set helping with all the in-time changes?
AE: You could run into a situation where I wouldn't say the shots wouldn't be possible to execute at all if you didn't have a visual FX representative on set, but you might not be able to execute them optimally. Because in post, you'd be guessing at a lot of things. For example, as I worked through the ranks, I was lighting, rendering, and compositing. I started my career as a CG generalist. I specialized for the middle of my career in lighting rendering and compositing, which is computer graphics CG. It’s possible to light something just by eye. I can reverse engineer by looking at an image and what the lighting was in the scene, and make a CG thing look mostly like that. But it's not quite the same as knowing with a high level of certainty where the lights were on the day on set. There are some things that you couldn't figure out if you didn't have on-set camera notes.
A typical Pixar movie takes about three years to make. Some of that is the story part of the process, so it's difficult to separate. In visual FX, part of the reason there are these mega visual FX studios that have a couple of thousand people is that they're working on big tent pole films that they only have about five months, four months to do a huge number of shots once the stuff gets turned over. You can load it up on lots of people and just do a lot of it in parallel. I think it's always a good idea.
There’s probably no director who wouldn’t love to have another pass on their edit. No writer wouldn't like to have the opportunity to tighten up the script, especially during shooting. This happens a lot more in TV. If we have more time to do it, having a smaller crew on staff tends to be better because the information is easier for people to get on the same page.
They're quick cycles, episodic stuff. It happens fast. We frequently need to get episodes done. We're lucky when we have a month and a half. Oftentimes, it's more like three weeks.
NMFO: Why don't you tell us a little bit more about Crafty Apes, the company that you work for? We know what you do within there. Tell us a little bit more about the company.
AE: Crafty Apes is a full-service visual FX studio. We have branch offices in multiple production hubs, including New Mexico. We have California, Vancouver, New Mexico, Baton Rouge, Atlanta, Montreal, and New York. And then we have a London office that's also starting now.
Overall, the company has around 500 people. If you want to do the overall headcount, we're a midsize visual FX studio.
NMFO: And why Albuquerque?
AE: Because it's one of the big production hubs. Now that we have Netflix and NBC, it made sense to have a branch here. We mostly cater to the New Mexico-based shows, but we also help some of the other branches with their shows. That's part of our strategy to be able to cater to shows in their local market so they can keep their claims process simple when using the state film incentives.
We are full service, which means we do both CGE and comp, and we also help with the planning, bids, breakdowns, and prep.
NMFO: You have tons of experience and a plethora of knowledge and training. How would you explain to somebody wanting to get into visual FX? What are ways to get involved in the industry?
AE: I like mentoring people, and I teach at UNM. As my responsibilities have increased lately, it's been harder for me. My class is still offered from time to time. So, if you're at UNM, I teach comp, but there are multiple specialties that you could do.
If you want to do visual FX, you don't really need certifications. You need to be able to produce awesome output. Now, having said that the tools are somewhat complicated, and there are a lot of possible distractions. So, that's one of the tricks of being able to really focus on the skills you need to develop. The first stage is figuring out which specialty you want to focus on to minimize the distractions of the other things you could be studying because there's almost an unlimited number of things you could learn in this business.
Then, start focusing on the specialties within visual FX that you want to practice and start building a very high level of skill around those. If you're interested in visual FX, I would say to learn some of the high-level, old-school techniques just to understand how they relate to modern techniques. That might help inform your decision of which specialty you want to do. Right now, we have a lot of shiny object syndrome. Filmmakers always want the coolest, neatest, shiniest toy to make their movies with. You don't need a techno crane to make a movie. You don't need a 70-millimeter Panavision to make a movie. You need a camera and maybe some lights, but if you're smart about it, you could shoot your whole movie using available light and a couple of flags, and it won't look ugly. It may not look as beautiful as something carefully handcrafted with many lights and instruments.
If you boil it down to the essence of what you need, you can do all the techniques essentially on your run-of-the-mill computer that you could buy at Best Buy or Costco. If you had a run-of-the-mill computer, it's not going to be the hottest, fastest workstation you could possibly have, but you could start doing work on it.
The barrier to entry is not high for getting started. Many software companies give away free or demo versions of their software. Usually, there are restrictions on how you can use it. You can't use it for commercial work. But for learning, it's perfectly fine. There's also open-source software, like Blender, you can use where the license is essentially free.
I would say phase one is figuring out what you want to specialize in. Then, once you've decided, you could start thinking about whether you want to pay for trade school. Some great trade schools have certificate programs. If you're going to get a degree, get a degree in something more general like filmmaking. Or, if you want to be doing visual FX, you might want to consider something more technical, like a STEM degree. It sounds crazy, but understanding computer science wouldn't hurt.
Then, do training using non-certificate or certificate programs where you can. All while practicing visual FX and film all through your training journey. Even while you're still in high school, if you look at some filmmakers that are successful, like Steven Spielberg, he was making movies in Super Eight when he was a child.
There's no barrier to entry. Even a child today has more access to filmmaking than Steven Spielberg did with 8mm. 8mm film is expensive. You can get a hand-me-down smartphone that has a better camera than a Super 8 camera ever was and start making film. You can even use the free or not expensive, editing app on the phone. You don't need additional support.
You can start learning the fundamentals of filmmaking before you even go to school. All that stuff is foundational and important to be a good visual FX person. You have to understand the filmmaking process on a slightly deeper level because if you can mess up the tone of a scene by how you complete a shot, you could make something that was supposed to be scary and create a sense of peril, you could make it into something that feels comedic, or not scary, just by the way you executed the visual FX shot.
You must understand that none of the general education regarding film theory will hurt your journey toward doing visual FX. It will help you understand the context of the visual FX and why they're going into the film. Maybe you're motivated as a filmmaker to learn visual FX simply as an optimization exercise for making your own movies better. It is a power tool if you can execute these things yourselves as an independent filmmaker or as a student, to be able to do little visual FX in your own projects. So, at the end of the day, your goal is, as a student, to be able to create a demo reel that can get you work.
That's your certification. Can you produce the output necessary to get a job? Where’s the bar for that? It's simple. What do you see on TV? What do you see in movies? Now, you're not going to be making Avatar in your bedroom. You just must pick your battles and decide, “Okay, what do we want to do here?”
Experiment, experiment basically,
NMFO: Figure out what brings you the most passion because, again, that's what you'll want to continue to pursue, learn more, and expand on.
AE: And don't allow what you perceive to be limitations. Don't assume that those limitations exist. Try to figure out how to circumvent those perceived limitations. As I said, these days, you can use an off-the-shelf, relatively modern commodity computer. It doesn't have to be a super workstation. There are tons of good print books that document the old traditional techniques and many new modern techniques. So, once you've got an idea of the direction you want to go, start with a general book, and then start getting the more focused books, and then as you start to learn, you'll realize quickly, “Okay, I'm working in a vacuum now. I need to be around more people. I need more input from other peers and mentors.” Then you'll realize, “Okay, maybe I need to go to a trade school.” Or at least they need to find some more local collaborators who are interested in the same stuff or online collaborators. There are many forums, too, where you can find moral support and get feedback. Just be careful because sometimes it can be the blind leading the blind.
NMFO: How can people get to know you, or Crafty Apes, more? Or learn about your mentorship programs or your class?
AE: If you want to see more about Crafty Apes, you can visit our website, craftyapes.com. You can check out our most recent demo reel.
I have a YouTube channel called LearnVFX. I have three free compositing videos right now. At UNM, I also teach in a program called FDM, Film and Digital Animation.