`unit-test` builder with `vitest` is slow for testing and for code coverage

[cexbrayat]

Command

test

Is this a regression?

• Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was

No response

Description

I migrated a very small existing project using the jasmine-to-vitest schematic and code coverage is fairly slow when using vitest (roughly 80 tests).

With the karma builder:

ng test --no-watch                               9,99s user 1,71s system 214% cpu 5,459 total
ng test --no-watch --code-coverage  17,82s user 2,42s system 232% cpu 8,719 total

When migrates to the unit-test builder (and a few tests disabled):

ng test --no-watch                     18,49s user 3,33s system 259% cpu 8,396 total
ng test --no-watch --coverage  38,46s user 4,66s system 210% cpu 20,535 total

The tests themselves takes 18s (10s with Karma), and the code coverage takes 20s instead of 8s. This is very noticeable when using vitest.

Minimal Reproduction

This is probably reproducible in a new CLI project, by generating a bunch of components and comparing test times.

Exception or Error

Your Environment

Angular CLI       : 21.0.0-rc.1
Angular           : 21.0.0-rc.1
Node.js           : 22.18.0
Package Manager   : npm 10.9.0
Operating System  : darwin arm64

┌───────────────────────────┬───────────────────┬───────────────────┐
│ Package                   │ Installed Version │ Requested Version │
├───────────────────────────┼───────────────────┼───────────────────┤
│ @angular/build            │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/cli              │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/common           │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/compiler         │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/compiler-cli     │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/core             │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/forms            │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/localize         │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/platform-browser │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ @angular/router           │ 21.0.0-rc.1       │ 21.0.0-rc.1       │
│ rxjs                      │ 7.8.2             │ 7.8.2             │
│ typescript                │ 5.9.3             │ 5.9.3             │
│ vitest                    │ 4.0.8             │ 4.0.8             │

Anything else relevant?

Of course this is maybe an upstream problem, but I thought it was worth mentioning it.

[clydin]

Vitest's test isolation improves reliability of tests but does have a cost. Also the file parallelism allows tests suites to be run in parallel but has a startup time cost per file.

Additionally, for code coverage, Vitest uses an AST based remapping system that provides excellent results but which can add to coverage analysis time.

Can you provide the breakdown of build vs test times for each scenario? Also relevant, builds will cache intermediate information by default which can greatly affect the overall process time. If not disabled, it might be useful to do so for profiling purposes.

Also, DOM emulation (jsdom/etc.) does also have a startup time cost. There are tradeoffs there especially when considering size of each test file and amount of test files.

You should also be able to experiment with some of these options via a custom runner config (runnerConfig option) similar to the following (vitest-base.config.ts):

import { defineConfig } from 'vitest/config';

export default defineConfig({
  test: {
    isolate: false,
    fileParallelism: false,
    environment: 'happy-dom',
  }

});

[cexbrayat]

I forgot to mention that I used ChromiumHeadless for the vitest runner.

Here are the build/test/coverage time.
Build took ~3.5s in all cases.

Karma:

ng test --no-watch  10,04s user 1,78s system 216% cpu 5,452 total
ng test --no-watch --code-coverage  13,16s user 2,02s system 256% cpu 5,921 total

Vitest (ChromiumHeadless):

Duration  3.17s (transform 0ms, setup 5.33s, collect 3.53s, tests 679ms, environment 0ms, prepare 40.31s)
ng test --no-watch  18,85s user 3,24s system 293% cpu 7,518 total

Duration  15.69s (transform 0ms, setup 6.35s, collect 4.07s, tests 805ms, environment 0ms, prepare 61.69s)
ng test --no-watch --coverage  38,67s user 4,71s system 214% cpu 20,193 total

Vitest (ChromiumHeadless with isolate: false) throws with NG0400 as @jnizet mentioned below, I can retry when the issue is fixed.

Vitest (ChromiumHeadless with fileParallelism: false):

Duration  3.37s (transform 0ms, setup 886ms, collect 419ms, tests 355ms, environment 0ms, prepare 48.71s)
ng test --no-watch  14,45s user 1,95s system 219% cpu 7,467 total

Duration  19.25s (transform 0ms, setup 894ms, collect 421ms, tests 431ms, environment 0ms, prepare 116.37s)
ng test --no-watch --coverage  32,47s user 2,83s system 150% cpu 23,456 total

I don't see a difference with this option.

Is there something else I can try/provide?

[jnizet]

Some additional numbers and information that might be interesting.

I ran a test on an app where I made 1000 copies of the same component (a form) (without removing the other ones). That makes 9139 tests in total.

This app was originally tested with Karma / Jasmine.

I rewrote the tests to use Vitest browser mode (playwright), with the Vitest browser API to interact with locators, retried assertions ans so on. So of course those tests do more than the original tests: they query by role and text, interact with the fields in a more realistic way, etc. I expect them to be a bit slower. But I consider these to be the best practices.

All tests (Karma and Vitest) are run in headless mode (ChromeHeadless for Karma, ChromiumHeadless for Vitest).

When I ran the original Jasmine tests with Karma, it took 17 min. and 18 seconds. But something was odd: the firsts tests go very fast (1000 tests in 23 seconds), but in the end, everything was very slow. The culprit was the default reporter configured by the CLI, which, I guess, clutters the DOM of the testing page with more and more elements.

Running the same tests with the progress reporter takes 3 minutes and 24 seconds (testing time only, reported by Karma).

Running the tests migrated to Vitest browser mode, on my laptop (MacBook Pro, 64 GB RAM, Intel 8 cores), takes 7 min. 18 seconds. That's really really slower, considering that Vitest runs the tests in parallel, and my machine has 8 cores.

This shows when running the tests with ``fileParallelism: false`: it takes 30 minutes and 17 seconds.

Trying to run them with isolate: false crashes the execution after 1000 tests (or so) with the following error:

Error: Failed to import test file /Users/jb/projects/books2/frontend/init-testbed.js
Caused by: Error: NG0400: A platform with a different configuration has been created. Please destroy it first.

Trying to run them with isolate: false and fileParallelism: false crashes the execution after 1 test with the same error:

Error: Failed to import test file /Users/jb/projects/books2/frontend/init-testbed.js
Caused by: Error: NG0400: A platform with a different configuration has been created. Please destroy it first.

Given that tests run fine with Karma without any isolation, I guess they should run without isolation in Vitest too, but that doesn't seem to be the case (see the error above).

Ideally, Vitest could run the tests in parallel (in N different iframes), but without isolation in each of these iframes. Then I guess we could have tests that take about the same time as with Karma, while still benefitting from the advantages and more realistic behavior of Vitest.

Addendum:

I migrated the duplicated test to Vitest, without switching to locators, i.e. keeping basically the same code as before. Running the 9009 tests with Vitest with fileParallelism: false leads to an execution time of 5 min. 27 seconds.
This shows that isolation, in browser mode, is costly (around 60% of overhead).

If Angular could make it possible to run tests without isolation (as in Karma), it would have a really nice effect on the test execution time.

[yjaaidi]

It is indeed way more performant without isolation. Iframe reload costs around 100ms per test file.

@clydin, I don't remember where I mentioned this but the init should be ran once in the virtual init file with something like:

const testBed = getTestBed();
if (testBed.platform) {
  return;
}

testBed.initTestEnvironment(...)

Workaround

In the meantime, it works with the analog plugin and we get really outstanding performance.

https://analogjs.org/docs/features/testing/vitest

[cexbrayat]

I retried with isolate: false and I have the same issue 👍

[yjaaidi]

It's fixed here: #31731
In the mean time you could also add a test setup file with the following workaround:

import { getTestBed } from '@angular/core/testing';
import { afterAll } from 'vitest';

afterAll(() => getTestBed().resetTestEnvironment());

Note that this workaround will be a tiny bit slower than the fix as this recreates a new test bed env for each test file

[yjaaidi]

There is an issue where setup files end up using dependencies duplicate modules when using Vitest with the CLI. This means two instances of TestBed. #31732

This only happens in browser mode and the workaround is to mark such dependencies as external dependencies:

1. create a custom build configuration in angular.json and add the following option "externalDependencies": ["@angular/core", "@angular/core/testing"]
2. update unit-test configuration to use "buildTarget": "build:test"

"build": {
  "builder": "@angular/build:application",
  "options": {...},
  "configurations": {
    ...,
    "test": {
      "optimization": false,
      "extractLicenses": false,
      "sourceMap": true,
      "externalDependencies": ["@angular/core", "@angular/core/testing"]
    }
  }
},
"test": {
  "builder": "@angular/build:unit-test",
  "options": {
    "buildTarget": ":build:test",
    "runner": "vitest",
    ...
  }
}