TLDR; Engineering for Creativity is about focusing on the who and not the what, building tools and not collectables, understanding the difference between correct and right, embracing constraints, shutting up and trying it, engaging with individuals, and being comfortable with constructive conflict. It is a unique challenge but it is a challenge that is worth it.

After weeks of investigation and design today was the day we present one of the final prototypes to the creative and engineering teams. The countless meetings, broken prototypes, hard feedback, and a lot of coffee laced evenings, had led to us extracting the last bits of feedback before we start the push to production. I was nervous, excited, and determined to get this right for the teams, both engineering and creative.

After everyone shuffles in, I begin my demo with the prototype that myself a few ambitious engineers helped me slap together. It is not even close to the final product but it has all the pieces to convey the idea. I finish the demo and the room is silent for a beat and I resist the urge to fill the void. Finally a creative supervisor speaks up,

"Thanks for putting this together, I have a few thoughts.",

Here. We. Go.

"I still don't understand why can't we can have feature X we talked about last time."

I am prepared to answer this but before I do an engineer on the team who will likely take this work on jumps in,

"We can't do that, it completely breaks the design of the system and conflicts with our integrations, it means users can do things we don't want them to do."

Another creative supervisor chimes in,

"I understand, but thats how we do this now without this tool and removing it means we have to change our process."

And without missing a beat the engineering lead responds,

"Yes, but your process is incorrect".

The debate continues to escalate and derail the meeting and as the time runs out and the meeting ends, I am left horrified that all that hard work is destroyed in the matter of 45 minutes and I have learn yet another important lesson in the dichotomy of engineering for creativity.

Stories like this are so common in my career that keeping count is impossible and it is easier to count the lessons I have learned from them. I am sure anyone who has had to mix an elixir of the proverbial oil and water, that is some creative team and some engineering team, together into a result has stories just like mine.

Over the years, I have spent a lot of days learning the hard lessons of trying to design and engineer solutions for creative endeavors; Watched egos clash, seen the best of intentions misunderstood, and good ideas from quiet voices shattered mercilessly. There are good parts as well; I've watched the magic of engineering empowering creativity, seen people lay their ego and pride aside for the betterment of the product, and witnessed the quiet voices rise to the top.

The dichotomy of engineering (and designing) for creativity is where I love to work; At the intersection of engineering's dependence on rigorous and correct process and creativity's inherent lack of process and boundless nature.

Engineering for Creativity

There is nothing quite like watching a user use a tool or system you built to create something beautiful, useful, and meaningful. It's one of the reasons I switch from being a (mostly terrible) artist to being an engineer and now product designer. I was in love with thinking about the parts and design of the tools and systems and how a small improvement can make a big difference.

However, as I progressed in my career and in my craft, there were more than a few times when I was reminded that I am engineering for creativity and not engineering creativity. The difference is subtle but important.

There are times when we are asked to build something and all the things we thought would make the lives of our users better and easier did not. There may even be times when the thing built was a net negative. What happened? Poor engineering? Bad design? If its my design, probably, but for everyone else its most likely we were too focused on the thing we were building rather than who we were building it for.

If you are engineer, you are definitely guilty of this and also know people guilty of this (like me!). Sometimes it so fun to just build, the things we are learning, trying, and designing are just so interesting and delightful that we forget to look up and see if we are even going in the right direction! It is 100% natural. Engineers love to learn and build, it's our fascination with understanding the details of how things work that makes us happy.

We do need to be careful. It is easy to get lost in how "cool" the thing we are building is and find ourselves building a framework or suite of tools when all we needed was a simple script. But how do we spot this? As an engineer, a lead, or a manager you should be asking "When was the last time we talked about and with the people who will use this?".

A lot engineers will avoid the user until "its ready" but that is a mistake. Keep that feedback loop small and tight. Outside of the usual "its easier to pivot", "user focused design", and "agile, fail fast" reasons, it more about keeping your ears open. A lot of folks, my younger self included, think you have to have the well organized prototype to get feedback, this is a false sense. You can get feedback on a napkin sketch, a quick conversation, or a half baked prototype. The early feedback is ambiguous, confusing, and sometimes contradictory but that is the point, we are wading through the unknowns and what ifs to find the right way.

Another way to put this is do not over-engineer but I have always found that to be un-grokable to me. My idea of simple maybe different from others. A 747's cockpit is simple for a pilot but utter nonsense to me but does that mean it is over-engineered? No. It just not engineered for me.

Collectables & Tools

Have you ever met someone who is a avid collector of shoes or something like that? The kind of person who has such an immense appreciation for the design and details of a thing that they will never use it or unwrap it? Engineers can tend to be like that at times with their design. I blame things like the SOLID principles, DRY and the like.

I remember reading "Clean Code" and being blown away at the ideas. It all made sense to me, to make great software we have to write great code. It's like finishing the back of the cabinet even though no one will see it! Oh, how times have changed. If you mention to me SOLID Principles you will have to forgive the knee jerk sigh I will let out. Don't get me wrong, I think they have their place and anything that emphasizes excellence in craftsmanship is good by me! But there is a difference between being a zealot and being a craftsman.

Throwing slag at Uncle Bob and his passionate Bob-ites is not the point here. Principles and patterns should be harnessed and learned but its when principle becomes religious dogma and gospel that we have a problem.

There were moments in my career where the things I was building were collectables not tools. Sometimes, as engineers and designers, we get so caught up in building the perfect thing that we forget we are building a tool. This leads to really rigid stances on things just because it may make the thing "unclean".

One of my favorite tools is my Apple Pencil. I live by this thing. I think its one of the most beautiful tools I own and its well designed (the magnetic one not the butt plugging one). But I can't help but imagine what the conversations were like around making the one side flat so it could attach magnetically to my iPad. Can you imagine if they decided "no, it has to be round, thats our design principle"? It would be a worse tool for it but probably a very nice collectable.

Our effort should always be towards building tools knowing we may have to compromise "properness" for the sake of "experience". It is a hard line to walk because we don't want to engineer a mess but we also don't want to engineer a perfect thing that can't get broken, dirty, amended, or used.

Some of the best tools I have seen created are the ones at first glance make you wince and question the skill set of the creator. But low and behold, it works. It may have started with some grand design but as time went on it evolved and now it's not the most elegant design but it is the most useful.

I have had to unlearn some of my "best practices" and "principles" dogma in order to temper it with reality and necessity. Making collectables is really fun but a collectable is just that, something to admire from a distance and that is not what we are about.

Correct is not always right

Full disclosure, I am the most incorrect person you will meet. I am wrong almost everyday of my life and I have put the phrase "two wrongs don't make a right" to the test many times. I am stubborn and strong willed and struggle to even stumble on the correct idea. This makes me a terrible engineer, a decent person to have a beer with, and a pretty good designer.

One moment in my career, I was employed as a FX Artist working on this shot to add some debris to a stereo rendered shot. I remember making the debris rig and pumping out a bunch or renders before review to get some feedback. I knew the debris's movement was not physically correct and the lighting was a bit off but I just wanted to get some direction and go from there.

"Approved! Looks Great!"

What the hell just happened. The debris was not done, it was so bad and I knew it but here I was with a "Great Job" from the supervisor. I was so confused and I remember asking him about it and asking him if I could make is correct. I explained how the physics wasn't right and the debris needs to bounce and roll more. He said something that has stuck with me since,

"Correct is not always right."

I wish I could say, at the time, I internalized that wisdom and politely thanked him for his time. I wish I could say I did not proceed to spend another 3 days on the shot only to have the next review of that new "correct" go something like "Why am I seeing this again? v2? Skip it, its approved by the client already.". But I can't say that.

Later on, this wisdom would pop back into my brain as I watched an engineer argue about the correctness of the solution with a stakeholder who was trying to explain why it didn't have to be "physically correct" but it just had to "look right".

The are moments we have to be willing to dial back our desire and predisposition for correctness and embrace the rightness of the solution. This not only applies to mathematical and systematic correctness but also to the correctness of the process and design. There are times when the correct way is not the right way.

Constraints are tools for creativity

Up until this point, it has been a bit of a slap on the wrist for some engineers and the creative folk are probably feeling pretty smug. Whelp you underpaid pixel pushers, its your turn!

Creativity is inherent to its very lack of process and bounds. The most creative people I know are incredibly exploratory, open, and willing to fail and learn. But creatives can be down right delusional in their desire for boundless creative freedom and lack of constraints.

I remember working as a FX Pipeline TD and having a debate with an artist about the simulation time constraints we put on the our render farm. This person was very irritated that I would not let his simulation run for 4 days straight, an increase of almost 500% from the 18hr limit. This person was a bit of a jerk about it but I knew, due to resource constraints, that I could not let it happen. The conversation ended and they left in a huff, and I moved on.

A few days later, I saw their work in review, and it was amazing. As they were reviewing the work the artist explained that their first iteration took too long to simulate and they had to rethink their approach, this led them to a more creative approach and it let them get the whole thing done in 2 hours. At the time I felt a bit bad for causing all that stress and trouble but it took a co-worker to point that the constraints we put on them forced them towards a better and more attractive solution. It was a hard pill to swallow but it was true.

Creatives love boundless freedom; It's the reason that, despite computer hardware and systems get faster each year, our render times over the past decade have only gotten longer. Creatives will always fill to the bounds of their arena but those constraints are tools for generating more creativity.

It is a balance, we have to be careful to not over constrain and stifle the process. I have born witness to the damage that causes and I will warn you that you do not get that creativity back easily; Like a fire, if you choke out it out with too much wood, too small a space, or too little air, you will have to restart it.

Shut up and F***ing Try It

I had been working on a new system for capturing high velocity statistics. The aim was to complement an existing system that was really awesome for capturing great analytics but it was a big thing to attach to your tools and programs and what was being asked of me was to make a system that could capture high volumes of data quickly with limited storage needs and performance impact.

It was time to demo and present it. The system was not revolutionary, a integration of a already existing ideas plus some special sauce for our business needs. While it was analogous to the other system, it served a different need.

The presentation went well and it was well received, or so I thought. For the next few months, I would hear rumbles about a few people who were not happy that we built this new thing. After a few more rumbles, I decided we needed to talk.

That conversation was a bit of a mess. I was trying to understand the problem they felt and they were trying to persuade me that the system was an affront to their principles and something needed to be done. I did my best to address and make clear the distinctions between the systems and address their problems and after while I was getting frustrated. With each thing I addressed they explained how it would not work and how it was not right. They could be right I said but we are going to see how it goes. But they persist and at some point I broke,

"Shut up and F***ing Try It"

My meeting with manager following that exchange was not great but it was warranted. That was not a good way to handle it. But....it worked. Being a bit shaken from seeing me lose my cool the individual started using the system, and they kept using it and soon they became one of the biggest advocates for the system and improvements.

Now the moral here is not, "be a jerk and it will work out" but rather sometimes we can get really caught up in the theoretical and conceptual that we forget the shutting up and trying it out is the best way to see how we can make something better.

There have be many times where I have seen a creative get so caught up in the conceptual and theoretical aspect of a product's design that the best thing for us to do it to just try it out. This also goes for any idea and concept someone has. Be open and try! Most of the time, the worst we lose is a bit of time and learn way more than you can in front of a whiteboard or in a meeting.

The power of individuals

"Death by thousand paper cuts" is a apt description for how I feel about large group meetings and committees. I am not anti-democratic but I more pro "get shit done". At many places I have worked, the mode of discussion was a group meeting or committee where everyone would try and solve and design things. It is a horrible tool for trying to build things.

I have gotten a lot of inspiration about how to rally a group of people around an idea or product from learning how CEOs work with their boards and present ideas. When a great CEO goes into a board meeting, it does not matter what the agenda is, everyone there knows what they are about to hear because the CEO has gone to them one-by-one to discuss it and get feedback to refine it. I love this.

Lots of product designers, engineers, and creatives tend to pitch ideas like they are giving an keynote. While I think this has its place, in a creative environment where the concerns of individuals need to be heard it will always be the loudest voice in the room that gets the ear in those rooms and in a multi-timezone environment, someone is always left out.

Engaging with individuals is an amazing experience as a designer. You can pitch the idea just like you would to a group but then you are able to focus the idea more and more with each person you meet so by the time you are presenting to the actual group the idea is clear and concise and the conversations are more constructive and fluid.

Polishing Rocks with Feathers

I am not sure where I heard the phrase "You can't polish rocks with feathers" but it's stuck with me somehow. When you bring a cross-discipline team together there is going to be some disagreement and conflict with the ideas and direction. This is a good thing and a sign of a healthy team.

There is a desire to have a zero conflict and low friction environment in most places. An idea that we can all get along to work together and make everyone happy. We should absolutely strive to have a inclusive and supportive environment but it does not mean everyone is going to be happy all the time. Constructive conflict is apart of making things.

Making great things is really hard. At each step of the process there are different choices and paths that can be taken. Not everyone is going to agree and that is okay. As long as people are respectful, a little passionate dispute is healthy; It is a sign that people care and are invested in the project. You don't want a room full of people agreeing with each other all the time, the end result is always crap, I am certain thats how the Magic Mouse was made.

It will be uncomfortable and down right aggravating when dealing with conflict but remember that is a "you" issue. You control how you react and think about how you feel and being able to work through conflicts is an invaluable skill.

At the intersection of engineering and creativity, we have to be comfortable with the fact that we are going to have to argue and debate sometimes. Yes, it may get a bit heated, but at the end of the day you cannot polish rocks with feathers, you have to use other rocks. And no please don't carry rocks into meetings.

A false dichotomy

This whole article has been about the dichotomy of engineering and creativity but I have to admit that is a bit of a false dichotomy. The two are not mutually exclusive and in fact they are very much codependent. The best creative work I have seen is the result of a great engineering process and the best engineering I have seen is the result of a great creative process. Engineering is not void of creativity and creativity is not void of great process.

While there are moments of tension and conflict its important to remember all the things these to disciplines have in common. The passion to create and build things, their curious nature, their ability to see the world differently, and their desire to make something meaningful and brings these two seemingly distance things together in such wonderful ways.

To understand their differences is to appreciate their similarities and appreciate that feeling of trying to mix oil and water is just feeling we get when the creative and engineering processes are working together to balance each other out.