How do you size up a company doing DevOps? For people seeking their next IT role, it’s a crucial question. Whether you’re already proving yourself a DevOps star or just breaking into DevOps, you don’t want to sign on with an organization that’s limping along – or worse, pretending to do DevOps. You also need to gauge how well the IT team delivers on results to the business. So you should go to interviews armed with smart questions and background on red flags.
DevOps methodology and culture prizes speed, experimentation, and collaboration, all happening on cross-functional teams. Developing that kind of culture usually involves great change. When you’re interviewing, it’s important to uncover where the organization is in terms of progress with the process and culture change. “Knowing where the organization is on the DevOps journey is important to match up with what someone wants to do,” says Red Hat’s Matt Micene, who frequently writes and speaks on DevOps culture. “If you want a mature practice and the organization is just starting, this could cause overall angst to a job seeker and a hiring organization.”
[ Are you a DevOps job seeker or a hiring manager? Get our free ebook: The Ultimate DevOps Hiring Guide. ]
We gathered 10 pieces of expert advice on what to look for – and what to watch out for – in teams and leaders as you interview with and evaluate a DevOps-minded IT organization.
1. Is there a separate DevOps team?
Take stock of how the IT team is organized. Is there a special DevOps team and are some people called “DevOps engineers?” This is a divisive issue among DevOps experts. Some say it’s wise to start the DevOps culture transition with a dedicated team; others call that approach hogwash and say you must spark the culture change across the whole organization. The separate team can be a clue to an immature DevOps shop, says Chris McFadden, Vice President of Engineering and Operations at email provider SparkPost.
“A great ‘DevOps shop’ will not have a separate DevOps team and no-one will have DevOps in their job title,” McFadden says. “This is a mistake more mediocre organizations can make, usually after a company’s management decides they need DevOps but do not really understand what that actually means. DevOps is a way of doing things across development and operations and is not one team's job.” See our related article, DevOps lessons learned, for the opposite point of view and more background on this topic.
2. Discuss success metrics
What metrics should teams be using to measure results from DevOps work? Many teams start with metrics tied to improving speed – a core part of DevOps methodology. “A great DevOps organization knows exactly how long each part of the release process takes,” says Robert Reeves, co-founder and CTO at Datical. “They have numbers to detail their improvement. Also, they will have set goals for themselves: ‘We decreased lead time from 12 days to 6 days. We want to get that down to 1 hour but our next goal is 3 days.’ That’s a great DevOps organization and one you want to work for.”
Additionally, some of the most meaningful metrics tie to business outcomes, says Red Hat technology evangelist Gordon Haff. For example, a business goal may be to improve customer experience. In this case, a useful metric would be customer ticket volume, “a reasonable proxy for overall customer satisfaction, which, in turn, strongly affects higher-level (and highly valued) measures such as Net Promoter Score, the willingness of customers to recommend a company's products or services to others,” as Haff noted in his article, DevOps metrics: Are you measuring what matters?
Or as Micene sums it up, “DevOps isn't just about going haphazardly faster, but delivering value quicker. How they measure and manage that is key.”
If none of the group’s metrics are tied to business outcomes, that’s a red flag.
3. Take a hard look at your potential immediate manager
Look for evidence that this is an agile leader who gets things done, says Jonathan Feldman, CIO of the city of Asheville, NC. “Are there lots of little projects going on? Is the person talking about effective projects?”
You want to hear IT leaders talking about project success – rather than just code or process, Feldman says. It’s one mark of an IT leader who is in tune with the business and delivering results.
The other place you see evidence that this manager gets things done? The hiring process, Feldman says. If the IT group takes a month to get back to people, it’s not moving at agile speed. Feldman says people who interview with his IT team often get an offer within a day of the interview, but always within three to four days. (Take note, hiring managers: CIOs like Feldman are not willing to lose agile candidates to other organizations due to slow HR process.)
4. Look for automation
How does the IT team use automation in tasks such as provisioning and CI/CD? Forward-thinking Ops teams use automation to free up time to solve problems. Lack of automation may be another clue that the organization is early in its DevOps journey, or that the Dev and Ops teams aren’t collaborating well.
Red Hat Chief Technology Strategist E.G. Nadhan says, "A mediocre DevOps shop has tools sporadically applied to the business of IT." You won't see much automation in such a shop (and they're missing the culture piece.)
On the other hand, here's what Nadhan says you'll see in a great DevOps shop: Culture, process, people, and tools applied to the business of the enterprise.
5. Ask about how teams collaborate
“The best organizations are fully collaborative and have created processes that foster the continual and costing communications across teams and roles,” says John Purrier, CTO of software company Automic.
Not all teams work the same, of course, and collaboration ideas are evolving. While DevOps started off with a focus on the handoff between developers and the Ops team, some companies have taken that idea further: “It’s about enabling ops to provide an environment for developers, then get out of the way as much as possible,” notes Haff in his recent article, DevOps success: A New team model emerges.
At the end of the day, you want to know the teams are in sync. “At a great ‘DevOps shop’ there is shared responsibility and accountability for the customer experience between the developers and the operations team. There are also shared goals for delivering value and ensuring reliability of the product,” says SparkPost's McFadden.
6. Explore how IT trains up staff
The DevOps culture emphasizes constant experimentation and improvement. In that spirit, so too should DevOps practitioners strive for “continuous improvement and renewal professionally and personally,” wrote Red Hat’s Brian Gracely, director of OpenShift product strategy, in a recent blog, The 7 Habits of Highly Effective DevOps.
You may think of training in terms of courses or certifications, but Gracely encourages DevOps leaders to think of training as a constant process. For example, IT leaders can incent employees to go learn new skills via side projects or meetups, and then frequently share the knowledge with the team.
“The proper way to address the need for 'skills improvement' is not to think about it as 'training' (e.g. attend a course, get a certification), but rather to incorporate it into an actual work activity,” Gracely writes.
If the group you’re interviewing with is doing this, that’s a sign of a mature DevOps organization.
7. Ask what makes people stay
People who shine in DevOps environments have a powerful story to tell potential employers – and they’re in demand. They don’t have to stay put. Are people staying at the organization you're interviewing with for a good chunk of time, or is there a big turnover problem? IT leadership can be a key reason why people stay or go in a DevOps culture.
And it’s not easy. DevOps requires IT leaders to rethink risk and other longtime IT principles. For example, take requirements: “IT leaders can no longer be passive recipients of business requirements, but instead must take responsibility for business outcomes,” says Mark Schwartz, CIO of U.S. Citizenship and Immigration Services, in his recent article, DevOps requires dumping old IT leadership ideas.
8. Ask people about their first week on the job
“A great ‘DevOps shop’ will have new hires deploy changes to production within their first week,” says McFadden. “This is a good sign that the team has ample continuous integration and deployment infrastructure in place and there is little drama in deploying to production. A more mediocre organization will be very protective of who can deploy changes to production, and possibly even have a separate team that handles deployments.”
9. Search for potential mentors
Especially with DevOps, you want to join a team that has a cluster of expertise – a group of people from whom you have much to learn. “We’re all the combination of our mentors,” says Feldman. “They can’t mentor you if they’re not really good at it.”
“Agile leaders are ambitious,” he says. “They want to get stuff done. They want to solve problems. They don’t want to write software that meets a spec but doesn’t solve a problem.”
10. Don't leave the interview without: A strategic vision
You should hear IT leaders, not just the CIO, articulate a strategic vision for where the business is going. They should be able to explain IT’s role to play, in a realistic manner. “There’s some kind of framework being used for how the work will be accomplished in a rapid, meaningful way,” Feldman says. So listen to learn where the organization and IT team are headed.
And what should send you running for the door? Plans for big-bang, old school IT implementations that take years should ring your warning bell, Feldman says. That’s not the mark of a DevOps shop. It’s the opposite of speed and flexibility.
Comments
This is a great article - thank you.
Thanks for the feedback, Geoff. Any advice to add here?