A simple request, yet 3 hours later I’m still jumping through favourite songs and artists trying to find the perfect song. It’s hard, there is one song that for me is so perfect, and exactly matches my personal motivations for much of my career, but blink and you will miss the part that means so much.
The Song is Go! by Public Service Broadcasting. It uses actual radio footage of the Apollo 11 landings, pretty motivating stuff for anyone, but there is 14 lines of dialogue that may get passed over, that have such a story to tell.
[Duke] 12 02 alarm
[Duke] 60 seconds
[Bales] We’re… we’re GO on that, Flight
[Kranz] We’re GO on that alarm?
[Aldrin] 30 feet down, 2 and a half
[Bales] It’s… If it doesn’t reoccur, we’ll be GO
[Kranz] 30 seconds
[Aldrin] 12 01
[Bales] 12 01
[Duke] Roger, 12 01 alarm
[Kranz] 12 01 alarm
[Bales] Same type, we’re GO, Flight
[Kranz] Ok, we’re go
[Carlton] We’ve had shutdown
[Armstrong] Houston, Tranquility Base here, The Eagle has landed
What we have here is the Landing Module Guidance computer basically crashing 60 seconds before the most momentous moment in human history, yet they didn’t abort, they pushed on, and the rest is (literally) history.
So much went into that decision to push on, and light needs to be shone on some of the backroom staff who allowed them to succeed, which is what Seenit is all about.
Gene Kranz – Flight Director, the ultimate decision maker
Steve Bales – Guidance officer, in charge of the guidance computer
Charles Duke – Astronaut working CAPCOM on this mission, the voice of mission control
Buzz Aldrin – Astronaut in the lander, monitoring the Computer
Neil Armstrong - Astronaut in the lander and first man on the moon.
Missing from the radio;
Jack Garman – Computer Engineer
What you see, and what has always inspired me, is that these people got it so right, and so quickly, and that’s because of several things.
Trust – There is almost no back and forth between them in the 14 seconds between the 1202 alarm and the Go decision. Kranz trusts that Bales has the right answer, and Bales trusts that Garman is correct, and when Duke relays this to Aldrin in the Lander, he trusts he has the right answer. NASA were so worried about this bond between mission control and the lander that they used other Astronauts who had worked with the crew to be the voice of mission control.
Communications – They were all there at the crucial moment, with open lines where needed to get the info to the right people at the right time.
Practice and Review – They had run countless simulations of the landing. And not just the Astronauts, but the entire team. It was Garman who had suggested that they need to run some simulations as to how the Mission Control staff would respond to the computer errors, and just before launch Bales had aborted a simulation erroneously due to a similar error, but the washup afterwards had identified that this could be a non fatal error.
Documentation – Kranz had insisted that everyone write out on plexiglass all the errors that could happen, and if they were a go or abort. Because of this, Garman was able to respond to the error even before Bales had figured out who to ask.
At Seenit we may not be literally shooting for the moon, but that doesn’t stop us in the engineering team taking inspiration from this. Over the last 12 months we have refined our processes and working practices to help us aim for our Moon, a better, more resilient, more secure platform, as well as releasing the features to help you to use the platform to capture your video.
Trust – We have refined the release process from something that could cause some concern from our Customer Success team and might mean we would have to delay the release if a big project was running that day, to something that the whole company trusts, and means we can release new code almost weekly, with no interruption to the service.
Communication – This is a big one for Seenit, and we have refined how engineering communicate with the rest of the business so whether it’s some new features or if there are any service issues, then the rest of the company know we are on it, and where to ask any questions if they need more info.
Practice and Review – You never want to be touching a production environment unless you have tested it on a different environment first. We always try new things in test, and with modern technologies such as kubernetes and docker, it’s simple to deploy a new environment, practice whatever ever type of deployment you need to do, test it, then review and refine before rebuilding the environment and trying again, until you have it fully automated and ready for production.
Documentation – There is always room for better documentation, and we have begun to improve our API documentation using swagger, which has improved things no end. We now have policy documents that cover all the important areas of the platform, and we collaborate with the rest of the business using Confluence for our wiki. It may not be glamorous, but it is important, and that is why documentation is part of our definition of done.
There is one other thing that was absolutely crucial in making the right decision. For all the simulations they had always taken the mindset where abort was the default decision in many scenarios. It was through practice that they realised that they needed to change their mindset, to default to Go, but that’s not an easy thing to do when the whole world is watching. These guys were brave, not just the astronauts, but everyone involved, and it’s that bravery, backed by really knowing your stuff that led to the right decision to Go, and that is what I find so motivating and inspiring. Whilst the Astronauts may take all the glory, President Nixon said as he awarded Steven Bales the NASA Group Achievement Award on behalf of the whole team “This is the young man, when the computers seemed to be confused and when he could have said Stop, or when he could have said Wait, said, Go.” Now that is pretty motivating to me.