Cucumber Open team's goals for 2021
  July 22, 2021

One of the first tasks on my plate as I took my new role as co-lead of Cucumber Open earlier this year was to come up with some annual OKRs for our team - big goals to set our direction for the year, and to give the leadership team at SmartBear feedback about the return they’re getting on their investment.

After some research, discussions with a few key members of the community, and in collaboration with Aslak, Aurélien and the wider team here at SmartBear, we came up with this overall mission:

Mission: Nurture and enable the Cucumber community to solve its own problems whenever possible.​

As a team of two-and-a-half full-time employees working on a codebase of over 300k source lines of code, it's not just impossible for us to triage all the tickets, fix all the bugs and merge all the pull requests; it would be a flawed strategy.

Instead, we need to act as facilitators for the community, nourishing it and enabling it to grow and flourish.

With this in mind, we set ourselves two objectives to focus on for the year:

  • Improve our responsiveness to volunteers' GitHub contributions.​
  • Increase the breadth of the volunteer contributor community.

I'll focus on each of these objectives in turn, telling you about the key results we'll be aiming to see, and the activities we're planning to achieve those results.

Objective: Improve responsiveness to volunteers' GitHub contributions

Our initial instinct for this objective was to look for results that show we've improved the flow of issues into pull requests into releases, as would be typical for a commercial product. However, after consultation with the regular volunteer contributors, it became clear that we're operating in a different context:

  • Volunteers don't want to feel any pressure to close tickets. This isn't work.
  • On open source projects, consensus is vital when decisions need to be taken, whether that's about product scope and direction, or architectural decisions. This consensus-building process needs patience and time.

This means we may need to accept having higher Work In Progress (WIP) and longer cycle times than we might ideally like.

What is important, however, is feedback. When someone opens a new issue or submits a pull request, they should get feedback on their question or code in a timely manner. Similarly, the back-and-forth of conversations around an issue should be happenning at a regular cadence. When the conversation dries up, it’s time for us to step in and help things move towards a decision.

Rien, the lead developer for Cucumber-JVM and a key member of the community, shared his experience of contributing to JUnit 5. He said that while the feedback he received on his contributions wasn’t incredibly quick, it was consistent.

So, with that in mind, I created a report to look across the whole Cucumber organisation to show how many days it had been since there had been any activity on a ticket. When I ran the report on 18th May, we had 547 open issues across the org, the oldest of which had been open for almost ten years!

chart showing that roughly 50%25 of Cucumber's issues have seen no activity in the last 90 days

Overall, about half the issues (229) had some activity within the past quarter, while a quarter of them (128) had seen no activity in over a year.

This clearly gives us our first Key Result:

Key Result: Increase the percentage of issues with activity within the last 90 days from 50% to 100%

We think nobody should have to wait longer than three months for feedback. Ideally, we'll get much better than this.

Looking at this report has already helped us to start to clear out the back of Cucumber’s metaphorical closet, finding old git repos that needed to be archived, all of which helps make our GitHub organisation easier to navigate for new contributors.

Progress on this goal since May has been steady, but we're going to need to put some systematic changes in place to how we work in order to reach the goal, and keep it as an ongoing service level.

chart showing growth in percentage of tickets answered in past 90 days

Objective: Increase the breadth of the contributor community

The second objective we have is to increase the breadth of the contributor community. Here, we're focussing on two key results:

  • Increase the number of new contributors from 118 in 2020 to 130 in 2021​
  • Increase number of recent, regular contributors who are non-white or non-male from 0 to 2

We’re already lucky enough to have volunteer contributors leading the development of the supported Java/JVM, JavaScript/Typescript and Go flavours of Cucumber, but often that’s down to one key individual. This places pressure on those individuals, making them more likely to burn out. This “key person dependency” presents the overall project (and community) with tremendous risk.

Looking at the number of new contributors is not a perfect metric, but it has encouraged us to take up initiatives like improving our contributor documentation. It's safe to assume that improvements we make for the onboarding and developer experience of new contributors should also make life better for regular contributors too, making it a decent leading indicator.

graph of new contributors per year

We've gone from a peak of 155 new contributors in 2018 to 118 last year. So aiming for 130 this year seems realistic, if a bit of a strech on the current trend:

chart of new contributors per month in 2021

Increasing the diversity of our core team

Joining one of our regular Thursday contributor Zoom meetings, it’s plain to see that the core team of regular contributors is exclusively made up of white men. This is a well-documented problem in open source, where the privileged are the ones with spare time to write code and get themselves established as contributors and community leaders.

In certainly reflects a lack of breadth in our contributor community.

I’m sure this lack of diversity in the core team does not reflect the diversity of our user base. If, as suggested in this blog post, we view our contributor onboarding as a funnel, we seem to be losing the people of colour and the women somewhere along the way. We’ve made many inroads already to make our community more welcoming and inclusive, but we can always do more.

Last Friday, we ran the first of what I hope will be many "new contributor ensemble" events where we gather a group of developers who have never contributed to open source before, and we work together to make a new contribution. The event was a great success and we'll be doing another one soon. If you're interested in joining in, please head over to the Cucumber Community Slack and find us in the #new-contributors channel.