At Edify, one of our operating agreements is that we work in and around people's natural workflows. To accomplish this we are constantly prioritizing conversations with software engineers so we can deeply understand the different ways engineers work, learn, and prefer to use tools.
Talking to our customers is fundamental to our success.
I know and have worked with many software engineers. I've seen firsthand the effects of poor onboarding programs and lack of knowledge management but prior to joining Edify, I don't think I'd ever asked one of my Engineering Manager colleagues "What do you like about your onboarding program? ...what do you hate about it?" I didn't understand the problem of engineering onboarding.
A year ago, when we set out to build our onboarding bot eddy, we knew that building would be the riskiest and most expensive part. We wanted to create a solution that would reduce the onboarding burden on managers and teams, increase time-to-productivity for new hires, and create space for high-value human interactions. Rather than start by building, we started by talking to our customers. Then we kept talking to them.
We launched eddy to our first beta customers in August 2020, just four months after the first lines of code were written. We were able to be narrow in our product scope because by that time we deeply understood the problem. By having these conversations, we understood how eddy would be able to deliver value to our customers, and that helped us focus and make quick decisions.
Don't Build Your Product on Data Alone
When I was a graduate student in Anthropology, I spent some time wrestling with the problem of the high failure rate of so many infrastructure projects aimed at helping improve health and social outcomes in poor communities. I recall coming across the parable of an organization that built a well in a rural community with no access to clean water. With limited resources and a short timeline, the organization quickly hired water scientists and structural engineers, they identified land near the community with an excellent water table, and they built a well. However, once completed, no one in the community used the well, and the project managers were left scratching their heads.
You may have heard some version of this story before. Or you may have even built a product like this yourself. The process probably looked something like this:
The “if you build it they will come” approach rarely works outside of the movies. To be successful, most products need to deeply understand their customer and their customer's needs before they start building.
The problem was the water organization only looked at geological data. They didn't talk to the people they wanted to serve. There could be a range of agricultural, logistical, or cultural reasons their well placement wasn't ideal but they didn't know because they didn't ask what the local community wanted.
Why does this approach fail?
- You didn’t understand the problem as the customer
- The solution doesn’t solve the problem for the customer
- The solution creates a new problem for the customer
- Your target customers don’t think they have a problem
For Edify, the data around time-to-productivity, retention, and churn told us that onboarding was a problem but we needed to understand how that problem was understood and felt by our customers before we built our solution.
Edify’s process looked like this:
Getting to Know Your Customer's Problem
Problems are chameleon-like and they manifest themselves in different ways for different people, so it’s important to understand the type of problem you are solving, the underlying causes, and who you are solving for.
Talking to different potential users helped us to hear the many facets of the problem of engineering onboarding. Managers said maintaining updated onboarding plans was painful, that documentation was quickly out of date, and that they didn't have good ways to measure new hire progress. Engineering leaders told us about problems with churn, lagging team productivity for months after a new hire starts, and disparate experiences from team to team. New hires told us about confusion, not knowing who to ask questions, and feeling isolated from their new team.
Get Feedback Early and Often
Once you understand your problem, you probably have some ideas about how to solve it. But there is still a risk you will build the wrong solution. Validate that your solution is something that your customers will want to use.
We build lots of prototypes, test our solution assumptions with our prospective users, and throw out the things that don’t work. Last spring, I ran a month-long experiment with an engineering team where I pretended to be eddy - testing our assumptions about features, and trying different conversation patterns to see what worked and what didn’t. That experiment helped us understand that our first solution for helping teams to customize their onboarding plans didn’t work, so we didn’t build it.
Build a Solution but Don’t Stop Getting Feedback
Problems are not static so we are constantly looking for ways to get feedback from our customers, both on the features they are using today, and those we are prototyping to build. We are focusing our user research on the areas where building new products is risky.
We take time to talk to customers and prospective customers every day. Our team meets weekly to discuss what we’ve learned from these customer interactions, conceive of new ideas, and refine our understanding of priorities based on that feedback.
Our team at Edify invests just as much energy into understanding the engineering onboarding problem and solutions as we do building for it. Users are your best source of real-world, useable data. And there is always more to learn.