Table of Contents

  • Landscape of E2E testing frameworks
    • WebDriver-Based
    • Chrome DevTools Protocol
    • Flakiness vs. Cross-Browser
  • Why you should use Cypress
  • Why you should not use Cypress
  • Summary
  • Further Reading

On April 24th, Angular publicly marked their E2E testing tool, Protractor, as deprecated. It's not yet clear whether a replacement will be provided by the Angular team or if this responsibility will be shifted to the community. As of now, WebDriver.IO, TestCafé, and Cypress have all released schematics compatible with the Angular CLI.

This piece offers a concise comparison of the available E2E frameworks and makes the case for choosing Cypress. The next installment will guide you through the initial steps with Cypress.

For those who prefer a video format, a recording of my talk is provided below:

E2E Testing Framework Landscape

WebDriver-Based

The existing tools can be broadly classified into two categories: those built on WebDriver and those that utilize the Chrome DevTools Protocol (CDP).

Selenium, introduced in 2004, eventually merged with a comparable project named WebDriver. Despite its age, Selenium remains a dominant force, and Protractor essentially serves as a layer on top of it.

Selenium/WebDriver provides a unified API for browser automation. Each supported browser has a corresponding driver, which WebDriver leverages to achieve cross-browser compatibility. Consequently, you can execute identical tests across Safari, Chrome, mobile browsers, and more.

Eventually, WebDriver's core technology became a W3C standard. This enabled third-party vendors to develop their own frameworks using WebDriver.

The downside is that WebDriver-based tests are often labeled as flaky. Under identical conditions, repeated runs may yield different outcomes—sometimes passing, sometimes failing. This is detrimental to a testing framework, as you invest significant effort creating tests to ensure application stability, only to find the test results themselves are unreliable.

Chrome DevTools Protocol

Puppeteer and Playwright are two well-known alternatives. Instead of relying on WebDriver, they communicate directly with the browser through CDP. This offers superior control, leading to more consistent test results.

CDP works with Chromium-based browsers like Chrome and Edge. While Firefox also has CDP support, it doesn't enable the type of comprehensive cross-browser testing that WebDriver does.

Playwright is led by some of Puppeteer's original developers, making it a kind of reimagined version. It utilizes CDP but extends its reach to Safari by shipping with patched browsers.

WebdriverIO takes a hybrid approach, offering access to both CDP and WebDriver.

Where does Cypress fit in? Cypress uses CDP for managing test execution but not for the tests themselves. The tests run natively in the browser, eliminating any intermediary layer. Your test code gains direct DOM access instead of going through CDP or WebDriver. Test and application code become integrated.

Interestingly, this mirrors the approach Selenium took before merging with WebDriver. History appears to be repeating itself.

Flakiness vs. Cross-Browser

From a broad perspective, increasing abstraction between tests and the browser tends to increase flakiness. Conversely, abstraction eases cross-browser testing. Proximity to the browser enhances test stability, but it usually confines you to a single browser technology, such as Chromium.
It's possible that future browser support for CDP or the emerging WebDriver BiDi standard will shift the balance, but the landscape is still evolving.

Landscape of E2E Frameworks


Landscape of E2E Frameworks

Reasons to Choose Cypress

Primarily, Cypress avoids WebDriver entirely, executing tests directly in the browser. This approach results in stable and dependable tests—which is the core objective.

Secondly, the developer experience (DX) is exceptional. The Cypress team has prioritized ease of use, not just in the API but also in the tooling, community contributions, and especially the documentation.

So, Cypress surpasses Protractor. But why select it over other E2E libraries?

Angular developers have consistently valued the "opinionated framework" that selects tools for us. Even though Angular appears to have left the E2E position vacant, a major player has stepped in: Nx has been providing premier tools for Angular applications for years.

Nx wraps the CLI. Initially focused on MonoRepositories, it has evolved into an enhanced Angular CLI++. Personally, I would choose Nx for any new Angular project. It has no lock-in and minimal overhead, offering improved tooling while remaining compatible with the Angular CLI.

Significantly, Nx has already substituted Protractor with Cypress, ensuring seamless integration between Angular and Cypress.

Finally, community adoption speaks volumes. Before publicizing the deprecation, Angular surveyed its users about their E2E practices. The results showed 64% using Cypress—a clear majority. In politics, this would be called a landslide. Remember, this was when Protractor was still the official tool! Protractor received only 19%.

Usage of E2E frameworks among Angular Devs


Source: https://github.com/angular/protractor/issues/5502

Reasons to Avoid Cypress

Depending on your specific requirements, Cypress might not be suitable:

  • No cross-browser testing: If your e2e tests must run on Safari, other browsers, or mobile devices, you'll need a different tool. Cypress is not the right fit.
  • Switching domains during a test: Applications spanning multiple domains will encounter limitations with Cypress.
  • Switching tabs: This functionality is also unavailable.

These are logical outcomes of Cypress's architecture. Since tests run within the browser, they are lost when the URL or tab changes.

Summary

Cypress delivers stability and a superior developer experience. A majority of Angular developers already utilize it.
If you're still using Protractor for E2E, now is the moment to migrate! If you've never written E2E tests, give Cypress a try—you might be impressed by its simplicity.
Transitioning to Cypress will require rewriting your E2E tests. This can be done incrementally, as Cypress and Protractor can coexist within the same Angular project.
Keep in mind that Cypress isn't for everyone. If cross-browser support is non-negotiable, consider alternatives like WebDriverIO or the other tools mentioned earlier.

Outlook

In part two, we'll take our first steps with Cypress.

Further Reading