👋 Hi, this is Gergely with a 🔒 subscriber-only issue 🔒 of the Pragmatic Engineer Newsletter. In every issue, I cover challenges at Big Tech and startups through the lens of engineering managers and senior engineers. If you’ve been forwarded this email, you can subscribe here. Quality Assurance Across the Tech IndustryAn overview of Quality Assurance (QA) approaches at various companies, and a look at tech segments where QA is on the decline, and where it is holding strong.
We previously dived deep into how 7 Big Tech companies – Microsoft, Google, Meta, Apple, Amazon, Uber and Netflix – approach quality assurance, covering how Microsoft retired its “dedicated software engineer in test” (SDET) position in 2014, while Apple and Amazon retain a high focus on dedicated QA. Today, we look beyond Big Tech, and analyze details from 47 tech professionals at different tech companies, who responded to a survey I put out about how companies approach QA. Thank you to everyone who contributed! In this issue, we cover:
1. Different forms of QA: QA, SDET, QE, and TE rolesThe survey results contain interesting insights about QA-related roles, which is often associated with manual testing. At most places, this is what “dedicated QA team” means; people who manually test a product. This is not always the case, but QA can also focus on automation and is usually called something else, like SDET, QE, or TE. SDET, QE, TE
These terms describe roles focused on automated testing.At companies which do very little dedicated manual testing, dedicated roles for quality assurance are often called:
In these roles, the mandate is to support quality-related work. However, the role is not to test software products. The role is about automation, empowering engineering teams to do testing, or both. Common responsibilities include:
At companies with SDET, QA, or TE roles, the writing of automated tests is split between devs and SDETs/QEs. Often, there are significantly fewer SDETs and QEs than developers. Quality coaches to transition off QA teamsA lead engineer working in a quality coach role at an Australian multinational, shared how they used to be a “quality coach” until their company made software engineers responsible for testing products, and formed a small group to support them, comprising:
Although this approach isn’t mentioned by more survey respondents, it makes sense for engineers to own quality in a transition from QA doing so. The concept of “quality coaches” seems to mirror agile coaches, common at companies rolling out practices like Scrum. Just as the “agile coach” role is redundant after teams learn the practices, so should the role of “quality coach” slowly disappear as testing becomes part of how engineering teams operate. Indeed, the respondee former quality coach works as a software engineer. We previously covered how Big Tech runs projects and the curious absence of Scrum. This transition also allows the company to convert QA staff into software engineers, if they’re willing to do more automated testing and software development. 2. Companies with no dedicated QA rolesWe mention above that most teams and orgs at Big Tech have no dedicated QA roles, except for Apple and teams with hardware divisions, like Google and Meta. Broadly, there are plenty of businesses with no dedicated QA roles. Here are the most common types I’ve identified. Early-stage startups and small companiesUnsurprisingly, new companies just starting out rarely hire folks dedicated to quality. This is what a survey respondent who’s a founder at a 3-person SaaS startup producing document storage, says:
At a 20-person Series A startup, a PM says they have no QAs, and engineers focus on automation:
Hiring the first QAs as contractors seems to be common. A founder at a 20-person startup shares that they are hiring a freelance QA to help set up the company’s testing foundation. Mid-sized startups and companiesA product and design PM at a profitable, 200-person, Series A company describes their place’s approach to QA:
A chief engineer at a 150-person marketing agency shares that they’ve moved away from relying on QAs:
It’s neat to learn how this agency works; that testing is present, but that developers own it. From the chief engineer:
A 500-person retail and commerce company made product managers and developers responsible for the product’s quality, putting in place:
Surprisingly, a 600-person company with 50 software engineers that manufactures electric vehicle chargers, does not have a QA department, but does have test engineers. A cloud software engineer there says:
Publicly traded companiesWe previously covered how most of Big Tech approaches QA, often via automated testing. An engineering director at a 4,000-person, publicly traded company summarizes their approach to testing:
This approach chimes with how many large tech companies operate. 3. Indeed eliminated the QA role. Is quality down?Indeed is a job search platform and publicly traded company, employing more than 14,000 staff, of whom the company laid off 15% (2,200) in March 2023. The layoffs resulted in the elimination of the QA role, as most QA engineers were let go. Those not shown the door were given 6 months to pass an interview and transition to an SWE position. Since then, developers are responsible for all QA, manual testing, and automation. A software engineer at Indeed tells me they’ve seen the quality of testing plummet:
In fairness, it’s common for quality to dip after a role is suddenly eliminated, and a layoff. What tends to happen is engineers pick up new responsibilities, the testing strategy is adopted, and over the next few months, teams adapt. Without dedicated QA staff, you cannot operate as before! I expect Indeed to go through a similar transition. It’s not only Indeed eliminating QA roles. A tech company based in Taiwan has done large layoffs, transitioning most former SDETs to developers, a software engineer shares. My hunch is the engineering leadership at that company hoped to have their cake and eat it by reducing the cost of engineering, while keeping quality as high as before. But quality usually drops automatically if employers mandate engineers to do more testing without a well thought-through transition away from dedicated QA. 4. Where QA is present, but decliningAn interesting group are companies with dedicated, but shrinking, QA teams. Here are groups that came up in this survey... Subscribe to The Pragmatic Engineer to read the rest.Become a paying subscriber of The Pragmatic Engineer to get access to this post and other subscriber-only content. A subscription gets you:
|
Quality Assurance Across the Tech Industry
Subscribe to:
Post Comments (Atom)
The Weekender: Moon metaphors, procrastinating monks, and a visit to a cheese factory
What we’re reading, watching, and listening to this week ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ...
No comments:
Post a Comment