e2e: Next-Generation Open-Source Agentic Testing Framework for Web and Mobile

End-to-End (E2E) testing across modern web and mobile applications remains one of the most critical yet labor-intensive facets of software delivery, often plagued by fragile UI selectors and fragmented testing stacks. While recent natural language AI testing agents have promised resilience against UI drift, their adoption has been severely limited by sluggish execution speeds and recurring LLM token expenditures on every test run. To bridge this divide, TesterArmy has unveiled 'e2e', an open-source testing framework that unifies deterministic Playwright-style locators with autonomous agent steps, backed by an intelligent action-caching engine that replays verified steps without redundant model calls.

Image source: TesterArmy / GitHub

Unifying Natural Language Agent Steps with Deterministic Locators

The core architectural innovation of e2e lies in its complementary blend of flexible natural language instructions and strict code-level assertions. Developers describe a goal in natural language and an agent interacts with the app to complete it via steps like agent.act(), while exact results are checked in the same test using familiar Playwright-style locators and assertions.

// Sample hybrid e2e test suite
test("Verify product checkout and cart interaction", async ({ page, agent }) => {
  await page.goto("https://example.com/shop");

  // Autonomous agent step navigating complex interactive flows
  await agent.act("Search for wireless noise-canceling headphones and add the first result to the cart");

  // Deterministic Playwright-style assertion validating precise UI state
  await expect(page.getByRole("button", { name: "Cart (1)" })).toBeVisible();
});

Developers retain granular control over the balance between autonomous agent actions and traditional deterministic assertions. Everything from the agent to the execution engine can be customized to suit specific project architectures.

Action Caching Replay Mechanism and Model Utilization

To solve the challenges of recurring model costs and execution latency inherent in AI testing tools, e2e incorporates an action caching mechanism.

When agent steps pass verification, the framework allows them to replay from cache on subsequent runs without calling the model. This mechanism preserves high test execution speeds while eliminating redundant model invocations and token costs on repeated runs.

Furthermore, agent steps run directly on the AI model or subscription the user already pays for.

Unified Web, iOS, and Android Execution and Open-Source Licensing

The e2e framework provides cross-platform testing capabilities across both web and mobile environments from a single TypeScript framework.

1. Cross-Platform Scope

   • The same tests run seamlessly across web, iOS, and Android, eliminating fragmented test stacks across platforms.

2. Open-Source Licensing and Customization

   • e2e is an open-source TypeScript framework released under the permissive Apache-2.0 license.
   • Developers can customize anything from the agent to the underlying engine to meet custom testing workflows.

3. Execution Workflows and CI Integration

   • Initial or uncached agent steps execute using the user's existing AI model or subscription before caching takes effect for passing steps.
   • Built by TesterArmy, the framework aligns with platform capabilities that run natural language tests on web and mobile apps on every pull request or on a schedule.

Sources

• GitHub Repository: tester-army/e2e
• TesterArmy Blog: Introducing e2e: open source agentic testing for web, iOS, and Android