Skip to content

Commit fc9475f

Browse files
author
Nikita Filonov
committed
docs
1 parent 2829040 commit fc9475f

File tree

2 files changed

+16312
-1
lines changed

2 files changed

+16312
-1
lines changed

‎README.md‎

Lines changed: 389 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1,389 @@
1-
TODO
1+
# UI Coverage Tool
2+
3+
**UI Coverage Tool** is an innovative, no-overhead solution for tracking and visualizing UI test coverage — directly on
4+
your actual application, not static snapshots. The tool collects coverage during UI test execution and generates an
5+
interactive HTML report. This report embeds a live iframe of your application and overlays coverage data on top, letting
6+
you see exactly what was tested and how.
7+
8+
## Features
9+
10+
- **Live application preview:** The report displays a real iframe of your app, not static screenshots. You can explore
11+
any page and see which elements were interacted with, what actions were performed, and how often.
12+
- **Flexible frame filters:** Focus only on what matters — filter elements by specific actions (`CLICK`, `FILL`,
13+
`VISIBLE`, etc.), or action groups. Ideal for analyzing specific scenarios or regression areas.
14+
- **Custom highlight & badge colors:** Easily change the highlight and badge colors used in the iframe for different
15+
action types or UI states. Great for tailoring the report to your team's visual style or accessibility needs.
16+
- **No framework lock-in:** Works with any UI testing framework (Playwright, Selenium, etc.) by simply logging actions
17+
via the `track_coverage()` method.
18+
- **Element-level statistics:** View detailed statistics by selector: type of action, count of actions, and a timeline
19+
graph of coverage.
20+
- **Global history overview:** Track historical trends of total coverage and action types across time.
21+
- **Per-element timeline:** Dive deep into the history of interactions for each element — when and how it was used.
22+
- **Full element index:** Searchable table of all elements interacted with during tests, even if you're not sure where
23+
they are in the UI.
24+
- **Multi-app support:** Testing multiple domains? No problem. Just list your apps in the config — the report will let
25+
you switch between them.
26+
27+
## Table of Contents
28+
29+
- [Features](#features)
30+
- [Links](#links)
31+
- [Preview](#preview)
32+
- [Installation](#installation)
33+
- [Embedding the Agent Script](#embedding-the-agent-script)
34+
- [Usage](#usage)
35+
- [Playwright](#playwright)
36+
- [Selenium](#selenium)
37+
- [Coverage Report Generation](#coverage-report-generation)
38+
- [Configuration](#configuration)
39+
- [.env](#configuration-via-env)
40+
- [YAML](#configuration-via-yaml)
41+
- [JSON](#configuration-via-json)
42+
- [Reference](#configuration-reference)
43+
- [Command-Line Interface (CLI)](#command-line-interface-cli)
44+
45+
## Links
46+
47+
### Example Report
48+
49+
You can view an example of a coverage report generated by the
50+
tool [here](https://nikita-filonov.github.io/ui-coverage-tool/).
51+
52+
### Questions & Support
53+
54+
If you have any questions or need assistance, feel free to ask [@Nikita Filonov](https://t.me/sound_right).
55+
56+
## Preview
57+
58+
### Summary
59+
60+
![Summary](./docs/screenshots/summary.png "Summary")
61+
62+
### History
63+
64+
![History](./docs/screenshots/history.png "History")
65+
66+
### Frame
67+
68+
![Frame](./docs/screenshots/frame.png "Frame")
69+
70+
### Element Details
71+
72+
![Element Details](./docs/screenshots/element_details.png "Element Details")
73+
74+
### Elements Table
75+
76+
![Elements Table](./docs/screenshots/elements_table.png "Elements Table")
77+
78+
## Installation
79+
80+
Requires Python 3.11+
81+
82+
```shell
83+
pip install ui-coverage-tool
84+
```
85+
86+
## Embedding the Agent Script
87+
88+
To enable live interaction and visual highlighting in the report, you must embed the coverage agent into your
89+
application.
90+
91+
Add this to your HTML:
92+
93+
```html
94+
95+
<script src="https://nikita-filonov.github.io/ui-coverage-report/agent.global.js"></script>
96+
```
97+
98+
That’s it. No other setup required. Without this script, the coverage report will not be able to highlight elements.
99+
100+
## Usage
101+
102+
Below are examples of how to use the tool with two popular UI automation frameworks: `Playwright` and `Selenium`. In
103+
both cases, coverage data is automatically saved to the `./coverage-results` folder after each call to `track_coverage`.
104+
105+
### Playwright
106+
107+
```python
108+
from playwright.sync_api import sync_playwright
109+
110+
# Import the main components of the tool:
111+
# - UICoverageTracker — the main class for tracking coverage
112+
# - SelectorType — type of selector (CSS, XPATH)
113+
# - ActionType — type of action (CLICK, FILL, CHECK_VISIBLE, etc.)
114+
from ui_coverage_tool import UICoverageTracker, SelectorType, ActionType
115+
116+
# Create an instance of the tracker.
117+
# The `app` value should match the name in your UI_COVERAGE_APPS config.
118+
tracker = UICoverageTracker(app="my-ui-app")
119+
120+
with sync_playwright() as playwright:
121+
browser = playwright.chromium.launch()
122+
page = browser.new_page()
123+
page.goto("https://my-ui-app.com/login")
124+
125+
username_input = page.locator("#username-input")
126+
username_input.fill('user@example.com')
127+
128+
# Track this interaction with the tracker
129+
tracker.track_coverage(
130+
selector='#username-input', # The selector (CSS)
131+
action_type=ActionType.FILL, # The action type: FILL
132+
selector_type=SelectorType.CSS # The selector type: CSS
133+
)
134+
135+
login_button = page.locator('//button[@id="login-button"]')
136+
login_button.click()
137+
138+
# Track the click action with the tracker
139+
tracker.track_coverage(
140+
selector='//button[@id="login-button"]', # The selector (XPath)
141+
action_type=ActionType.CLICK, # The action type: CLICK
142+
selector_type=SelectorType.XPATH # The selector type: XPath
143+
)
144+
145+
```
146+
147+
Quick summary:
148+
149+
- Call `tracker.track_coverage()` after each user interaction.
150+
- Provide the selector, action type, and selector type.
151+
- The tool automatically stores tracking data as JSON files.
152+
153+
### Selenium
154+
155+
```python
156+
from selenium import webdriver
157+
from ui_coverage_tool import UICoverageTracker, SelectorType, ActionType
158+
159+
driver = webdriver.Chrome()
160+
161+
# Initialize the tracker with the app key
162+
tracker = UICoverageTracker(app="my-ui-app")
163+
164+
driver.get("https://my-ui-app.com/login")
165+
166+
username_input = driver.find_element("css selector", "#username-input")
167+
username_input.send_keys("user@example.com")
168+
169+
# Track the fill action
170+
tracker.track_coverage('#username-input', ActionType.FILL, SelectorType.CSS)
171+
172+
login_button = driver.find_element("xpath", '//button[@id="login-button"]')
173+
login_button.click()
174+
175+
# Track the click action
176+
tracker.track_coverage('//button[@id="login-button"]', ActionType.CLICK, SelectorType.XPATH)
177+
```
178+
179+
### Coverage Report Generation
180+
181+
After every call to `tracker.track_coverage(`), the tool automatically stores coverage data in the `./coverage-results/`
182+
directory as JSON files. You don’t need to manually manage the folder — it’s created and populated automatically.
183+
184+
```
185+
./coverage-results/
186+
├── 0a8b92e9-66e1-4c04-aa48-9c8ee28b99fa.json
187+
├── 0a235af0-67ae-4b62-a034-a0f551c9ebb5.json
188+
└── ...
189+
```
190+
191+
Once your tests are complete and coverage data has been collected, generate a final interactive report using this
192+
command:
193+
194+
```shell
195+
ui-coverage-tool save-report
196+
```
197+
198+
This will generate:
199+
200+
- `index.html` — a standalone HTML report that you can:
201+
- Open directly in your browser
202+
- Share with your team
203+
- Publish to GitHub Pages / GitLab Pages
204+
- `coverage-report.json` — a structured JSON report that can be used for:
205+
- Printing a coverage summary in CI/CD logs
206+
- Sending metrics to external systems
207+
- Custom integrations or dashboards
208+
209+
**Important!** The `ui-coverage-tool save-report` command must be run from the **root of your project**, where your
210+
config files (`.env`, `ui_coverage_config.yaml`, etc.) are located. Running it from another directory may result in
211+
missing data or an empty report.
212+
213+
## Configuration
214+
215+
You can configure the UI Coverage Tool using a single file: either a YAML, JSON, or `.env` file. By default, the
216+
tool looks for configuration in:
217+
218+
- `ui_coverage_config.yaml`
219+
- `ui_coverage_config.json`
220+
- `.env` (for environment variable configuration)
221+
222+
All paths are relative to the current working directory, and configuration is automatically loaded
223+
via [get_settings()](./ui_coverage_tool/config.py).
224+
225+
**Important!** Files must be in the project root.
226+
227+
### Configuration via `.env`
228+
229+
All settings can be declared using environment variables. Nested fields use dot notation, and all variables must be
230+
prefixed with `UI_COVERAGE_`.
231+
232+
**Example:** [.env](docs/configs/.env.example)
233+
234+
```dotenv
235+
# Define the applications that should be tracked. In the case of multiple apps, they can be added in a comma-separated list.
236+
UI_COVERAGE_APPS='[
237+
{
238+
"key": "my-ui-app",
239+
"url": "https://my-ui-app.com/login",
240+
"name": "My UI App",
241+
"tags": ["UI", "PRODUCTION"],
242+
"repository": "https://github.com/my-ui-app"
243+
}
244+
]'
245+
246+
# The directory where the coverage results will be saved.
247+
UI_COVERAGE_RESULTS_DIR="./coverage-results"
248+
249+
# The file that stores the history of coverage results.
250+
UI_COVERAGE_HISTORY_FILE="./coverage-history.json"
251+
252+
# The retention limit for the coverage history. It controls how many historical results to keep.
253+
UI_COVERAGE_HISTORY_RETENTION_LIMIT=30
254+
255+
# Optional file paths for the HTML and JSON reports.
256+
UI_COVERAGE_HTML_REPORT_FILE="./index.html"
257+
UI_COVERAGE_JSON_REPORT_FILE="./coverage-report.json"
258+
```
259+
260+
### Configuration via YAML
261+
262+
**Example:** [ui_coverage_config.yaml](docs/configs/ui_coverage_config.yaml)
263+
264+
```yaml
265+
apps:
266+
- key: "my-ui-app"
267+
url: "https://my-ui-app.com/login",
268+
name: "My UI App"
269+
tags: [ "UI", "PRODUCTION" ]
270+
repository: "https://github.com/my-ui-app"
271+
272+
results_dir: "./coverage-results"
273+
history_file: "./coverage-history.json"
274+
history_retention_limit: 30
275+
html_report_file: "./index.html"
276+
json_report_file: "./coverage-report.json"
277+
```
278+
279+
### Configuration via JSON
280+
281+
**Example:** [ui_coverage_config.json](docs/configs/ui_coverage_config.json)
282+
283+
```json
284+
{
285+
"services": [
286+
{
287+
"key": "my-ui-app",
288+
"url": "https://my-ui-app.com/login",
289+
"name": "My UI App",
290+
"tags": [
291+
"UI",
292+
"PRODUCTION"
293+
],
294+
"repository": "https://github.com/my-ui-app"
295+
}
296+
],
297+
"results_dir": "./coverage-results",
298+
"history_file": "./coverage-history.json",
299+
"history_retention_limit": 30,
300+
"html_report_file": "./index.html",
301+
"json_report_file": "./coverage-report.json"
302+
}
303+
```
304+
305+
### Configuration Reference
306+
307+
| Key | Description | Required | Default |
308+
|---------------------------|---------------------------------------------------------------------------|----------|---------------------------|
309+
| `apps` | List of applications to track. Each must define `key`, `name`, and `url`. | ✅ | — |
310+
| `services[].key` | Unique internal identifier for the service. | ✅ | — |
311+
| `services[].url` | Entry point URL of the app. | ✅ | — |
312+
| `services[].name` | Human-friendly name for the service (used in reports). | ✅ | — |
313+
| `services[].tags` | Optional tags used in reports for filtering or grouping. | ❌ | — |
314+
| `services[].repository` | Optional repository URL (will be shown in report). | ❌ | — |
315+
| `results_dir` | Directory to store raw coverage result files. | ❌ | `./coverage-results` |
316+
| `history_file` | File to store historical coverage data. | ❌ | `./coverage-history.json` |
317+
| `history_retention_limit` | Maximum number of historical entries to keep. | ❌ | `30` |
318+
| `html_report_file` | Path to save the final HTML report (if enabled). | ❌ | `./index.html` |
319+
| `json_report_file` | Path to save the raw JSON report (if enabled). | ❌ | `./coverage-report.json` |
320+
321+
### How It Works
322+
323+
Once configured, the tool automatically:
324+
325+
- Tracks test coverage during UI interactions.
326+
- Writes raw coverage data to `coverage-results/`.
327+
- Stores optional historical data and generates an HTML report at the end.
328+
329+
No manual data manipulation is required – the tool handles everything automatically based on your config.
330+
331+
## Command-Line Interface (CLI)
332+
333+
The UI Coverage Tool provides several CLI commands to help with managing and generating coverage reports.
334+
335+
### Command: `save-report`
336+
337+
Generates a detailed coverage report based on the collected result files. This command will process all the raw coverage
338+
data stored in the `coverage-results` directory and generate an HTML report.
339+
340+
**Usage:**
341+
342+
```shell
343+
ui-coverage-tool save-report
344+
```
345+
346+
- This is the main command to generate a coverage report. After executing UI tests and collecting coverage data, use
347+
this command to aggregate the results into a final report.
348+
- The report is saved as an HTML file, typically named index.html, which can be opened in any browser.
349+
350+
### Command: `copy-report`
351+
352+
This is an internal command mainly used during local development. It updates the report template for the generated
353+
coverage reports. It is typically used to ensure that the latest report template is available when you generate new
354+
reports.
355+
356+
**Usage:**
357+
358+
```shell
359+
ui-coverage-tool copy-report
360+
```
361+
362+
- This command updates the internal template used by the save-report command. It's useful if the template structure or
363+
styling has changed and you need the latest version for your reports.
364+
- This command is typically only used by developers working on the tool itself.
365+
366+
### Command: `print-config`
367+
368+
Prints the resolved configuration to the console. This can be useful for debugging or verifying that the configuration
369+
file has been loaded and parsed correctly.
370+
371+
**Usage:**
372+
373+
```shell
374+
ui-coverage-tool print-config
375+
```
376+
377+
- This command reads the configuration file (`ui_coverage_config.yaml`, `ui_coverage_config.json`, or `.env`)
378+
and prints the final configuration values to the console.
379+
- It helps verify that the correct settings are being applied and is particularly useful if something is not working as
380+
expected.
381+
382+
## Troubleshooting
383+
384+
### The report is empty or missing data
385+
386+
- Ensure that `track_coverage()` is called during your tests.
387+
- Make sure you run `ui-coverage-tool save-report` from the root directory.
388+
- Make sure to setup configuration correctly.
389+
- Check that the `coverage-results` directory contains `.json` files.

0 commit comments

Comments
 (0)