|
| 1 | +# End to end tests |
| 2 | + |
| 3 | +E2e tests largely follow the same syntax as [integration tests](tests/e2e/README.md). |
| 4 | +Whereas integration tests are intended to mock and stress the back-end, server-side code, e2e tests the interface between front-end and back-end, as well as visual regressions with both assertions and visual comparisons. |
| 5 | +They can be run with make commands for the appropriate backends, namely: |
| 6 | +```shell |
| 7 | +make test-sqlite |
| 8 | +make test-pgsql |
| 9 | +make test-mysql |
| 10 | +make test-mysql8 |
| 11 | +make test-mssql |
| 12 | +``` |
| 13 | + |
| 14 | +Make sure to perform a clean front-end build before running tests: |
| 15 | +``` |
| 16 | +make clean frontend |
| 17 | +``` |
| 18 | + |
| 19 | +## Install playwright dependencies |
| 20 | +``` |
| 21 | +npx install playwright --with-deps |
| 22 | +``` |
| 23 | + |
| 24 | + |
| 25 | +## Run all tests via local drone |
| 26 | +``` |
| 27 | +drone exec --local --build-event "pull_request" |
| 28 | +``` |
| 29 | + |
| 30 | +## Run sqlite e2e tests |
| 31 | +Start tests |
| 32 | +``` |
| 33 | +make NO_DEPS_PLAYWRIGHT=1 test-e2e-sqlite |
| 34 | +``` |
| 35 | + |
| 36 | +## Run MySQL e2e tests |
| 37 | +Setup a MySQL database inside docker |
| 38 | +``` |
| 39 | +docker run -e "MYSQL_DATABASE=test" -e "MYSQL_ALLOW_EMPTY_PASSWORD=yes" -p 3306:3306 --rm --name mysql mysql:latest #(just ctrl-c to stop db and clean the container) |
| 40 | +docker run -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" --rm --name elasticsearch elasticsearch:7.6.0 #(in a second terminal, just ctrl-c to stop db and clean the container) |
| 41 | +``` |
| 42 | +Start tests based on the database container |
| 43 | +``` |
| 44 | +TEST_MYSQL_HOST=localhost:3306 TEST_MYSQL_DBNAME=test TEST_MYSQL_USERNAME=root TEST_MYSQL_PASSWORD='' NO_DEPS_PLAYWRIGHT=1 make test-e2e-mysql |
| 45 | +``` |
| 46 | + |
| 47 | +## Run pgsql integration tests |
| 48 | +Setup a pgsql database inside docker |
| 49 | +``` |
| 50 | +docker run -e "POSTGRES_DB=test" -p 5432:5432 --rm --name pgsql postgres:latest #(just ctrl-c to stop db and clean the container) |
| 51 | +``` |
| 52 | +Start tests based on the database container |
| 53 | +``` |
| 54 | +TEST_PGSQL_HOST=localhost:5432 TEST_PGSQL_DBNAME=test TEST_PGSQL_USERNAME=postgres TEST_PGSQL_PASSWORD=postgres NO_DEPS_PLAYWRIGHT=1 make test-e2e-pgsql |
| 55 | +``` |
| 56 | + |
| 57 | +## Run mssql integration tests |
| 58 | +Setup a mssql database inside docker |
| 59 | +``` |
| 60 | +docker run -e "ACCEPT_EULA=Y" -e "MSSQL_PID=Standard" -e "SA_PASSWORD=MwantsaSecurePassword1" -p 1433:1433 --rm --name mssql microsoft/mssql-server-linux:latest #(just ctrl-c to stop db and clean the container) |
| 61 | +``` |
| 62 | +Start tests based on the database container |
| 63 | +``` |
| 64 | +TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=gitea_test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 NO_DEPS_PLAYWRIGHT=1 make test-e2e-mssql |
| 65 | +``` |
| 66 | + |
| 67 | +## Running individual tests |
| 68 | + |
| 69 | +Example command to run `example.test.e2e.js` test file: |
| 70 | + |
| 71 | +_Note: unlike integration tests, this filtering is at the file level, not function_ |
| 72 | + |
| 73 | +For SQLite: |
| 74 | + |
| 75 | +``` |
| 76 | +make test-e2e-sqlite#example |
| 77 | +``` |
| 78 | + |
| 79 | +For other databases(replace `mssql` to `mysql`, `mysql8` or `pgsql`): |
| 80 | + |
| 81 | +``` |
| 82 | +TEST_MSSQL_HOST=localhost:1433 TEST_MSSQL_DBNAME=test TEST_MSSQL_USERNAME=sa TEST_MSSQL_PASSWORD=MwantsaSecurePassword1 NO_DEPS_PLAYWRIGHT=1 make test-e2e-mssql#example |
| 83 | +``` |
| 84 | + |
| 85 | +## Visual testing |
| 86 | + |
| 87 | +Although the main goal of e2e is assertion testing, we have added a framework for visual regress testing. If you are working on front-end features, please use the following: |
| 88 | + - Check out `main`, `make clean frontend`, and run e2e tests with `VISUAL_TEST=1` to generate outputs. This will initially fail, as no screenshots exist. You can run the e2e tests again to assert it passes. |
| 89 | + - Check out your branch, `make clean frontend`, and run e2e tests with `VISUAL_TEST=1`. You should be able to assert you front-end changes don't break any other tests unintentionally. |
| 90 | + |
| 91 | +VISUAL_TEST=1 will create screenshots in tests/e2e/test-snapshots. The test will fail the first time this is enabled (until we get visual test image persistence figured out), because it will be testing against an empty screenshot folder. |
| 92 | + |
| 93 | +ACCEPT_VISUAL=1 will overwrite the snapshot images with new images. |
0 commit comments