Python Automation Framework with Playwright + Behave + Allure + AI

A powerful, scalable automation framework combining Playwright for browser automation, Behave for BDD testing, Allure for reporting, Page Object Model for maintainable test structure, and AI-powered selector healing for robust test automation.

🚀 Quick Start

# Clone and set up
git clone https://github.com/prashant1507/playwright-behave-allure-framework.git
cd playwright-behave-allure-framework/

# Install dependencies
pip install -r requirements.txt
playwright install

# Install Node for checking tracing
brew install node

# Run your first test
python run_tests.py --tags @smoke

✨ Key Features

Feature	Description	Benefits
🧠 AI Selector Healing	AI-powered selector recovery using Ollama	Self-healing tests, reduced maintenance
🏗️ Page Object Model	Centralized element selectors and reusable page methods	Maintainable, scalable test structure
🚀 Parallel Execution	Multi-worker test execution feature-by-feature	Faster test execution, efficient resource usage
🏷️ Smart Tag Filtering	Filter tests by tags (`@smoke`, `@regression`, `@api`)	Run only relevant tests, reduce execution time
📊 Enhanced Reporting	Allure integration with automatic screenshots	Detailed HTML reports with failure analysis
⚙️ Flexible Configuration	YAML config + environment variables + command-line args	Easy configuration management
🌐 Multi-Browser Support	Chromium, Firefox, WebKit support	Cross-browser testing capabilities
🎯 Clean Output	Filtered console output with organized reporting	Reduced noise, better debugging

🏗️ Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Feature Files │    │  Step Defs      │    │  Page Objects   │
│   (Gherkin)     │───▶│  (Test Logic)   │───▶│  (Selectors)    │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Behave        │    │   Playwright    │    │   Allure        │
│   (BDD Engine)  │    │   (Browser)     │    │   (Reporting)   │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   AI Selector   │    │   Ollama        │    │   Selector      │
│   Healer        │◀──▶│   (AI Model)    │───▶│   Map Cache     │
└─────────────────┘    └─────────────────┘    └─────────────────┘

🧠 AI Selector Healing

The framework includes an intelligent AI-powered selector healing system that automatically recovers from selector failures using the Ollama AI model.

How It Works

Automatic Detection: When a selector fails (throws an exception), the system automatically triggers AI healing
Context Capture: Captures current page screenshot and HTML content
AI Analysis: Uses Ollama (devstral:24b model) to analyze the page and suggest new selectors
Validation: Validates AI-suggested selectors before using them
Learning: Maintains a selector_map.json file for future reference

Features

🧠 Intelligent Recovery: AI analyzes page structure and suggests optimal selectors
📸 Visual Analysis: Uses screenshots for better element identification
🎯 Confidence Scoring: AI provides confidence levels for suggested selectors
📚 Historical Learning: Maintains selector mapping for reuse and learning
🔍 Multiple Selector Types: Supports XPath, CSS, and text-based selectors
⚡ Automatic Integration: Seamlessly integrated into Page Object Model

Example Usage

# In base_page.py - Automatic AI healing
def fill_input(self, selector: str, value: str):
    try:
        self.page.locator(selector).wait_for(timeout=5000)
        self.page.fill(selector, value)
    except playwright.sync_api.TimeoutError:
        # AI healing automatically triggered
        locator = ai_selector_healing(context=self.context, text=selector)
        locator.fill(value)

Benefits

🔄 Self-Healing Tests: Tests automatically recover from selector changes
📉 Reduced Maintenance: Less manual selector updates required
🎯 Higher Reliability: AI suggests robust, context-aware selectors
📚 Continuous Learning: Improves over time with historical data
⚡ Faster Development: Reduces debugging time for selector issues

Notes

The AI method is ready, and requires the user to adjust base_page.py functions
Use the @ai_healing tag to see AI in action

⚙️ Setup Guide

1. Prerequisites

Python 3.12+
Git (for version control)
Allure (for reporting)
Ollama (for AI models)

2. Environment Setup

# Create a virtual environment
python -m venv venv

# Activate virtual environment
# On macOS/Linux:
source venv/bin/activate
# On Windows:
venv\Scripts\activate

# Upgrade pip
pip install --upgrade pip

# Install Python packages
pip install -r requirements.txt

# Install Playwright browsers
playwright install

3. Install Allure

# macOS (using Homebrew)
brew install allure

# Windows (using Scoop)
scoop install allure

# Linux
sudo apt-add-repository ppa:qameta/allure
sudo apt-get update
sudo apt-get install allure

4. Install Ollama (Required for AI Selector Healing)

# Install Ollama
https://ollama.ai/

# Pull the required model
ollama pull devstral:24b

5. Configuration

Set URL in Config.yaml:

base_url: https://httpbin.org

Important: The base_url is required in config.yaml. The framework will raise an error if it's missing.

6. AI Selector Healing Configuration

The AI selector healing system is automatically configured and ready to use. It will:

Create selector_map.json for historical selector mapping
Generate selector_log.json for AI interaction logs
Capture screenshots in reports/screenshots/ai-*.png for AI analysis
Use the devstral:24b Ollama model by default

7. Verify Installation

# Run a quick test
python run_tests.py --tags @smoke --headless

Note: The AI selector healing will automatically activate when selectors fail. You can monitor AI interactions in the console output and check the generated logs.

🧪 Running Tests

Check Script Usage

    python3 run_tests.py --help

Basic Test Execution

# Run all tests
python run_tests.py

# Run specific feature files
python run_tests.py features/login.feature features/forms.feature

# Run tagged tests
python run_tests.py --tags @smoke

# Run with tracing
python run_tests.py --tracing

# Run with optimal worker count
python run_tests.py --parallel

# and so on

Advanced Combinations

# Parallel execution with a specific browser and headless mode
python run_tests.py --parallel --browser firefox --headless --workers 4

# Run specific tags with custom configuration
python run_tests.py --tags @smoke @api --browser webkit --headless

# Run with auto-serve report
python run_tests.py --tags @smoke --serve-report

📊 Reporting

Allure Reports

# Generate and serve a report
python run_tests.py --tags @smoke --serve-report

# Or manually serve the existing report
allure serve reports/allure-results

Report Features

📊 HTML Reports - Detailed test results with trends
📸 Screenshots - Automatic capture on failures
📋 Failing Scenarios - Clear summary of failed tests
📈 Trends - Historical test execution data
🔍 Detailed Analysis - Step-by-step failure analysis

Report Structure

reports/
├── allure-results/          # Allure report data
│   ├── *.json               # Test results
│   └── *.xml                # Test metadata
├── screenshots/             # Failure screenshots
│   └── screenshot_*.png     # Automatic screenshots
│   └── ai-*.png             # AI screenshots
└── workers/                 # Parallel execution logs
    └── worker_*.log         # Worker-specific logs

Best Practices

Keep selectors in page objects - Centralized element management
Keep assertions in step definitions - Test logic separation
Use Playwright's expect() - Reliable assertions
Create reusable page methods - Reduce code duplication

🏷️ Tag Filtering

Available tags in the framework:

@smoke - Quick validation tests
@regression - Comprehensive test suite
@api - API testing scenarios
@performance - Performance testing

⚙️ Code Organization

Keep step definitions focused

@when("the user clicks the login button")
def step_click_login(context):
    login_page = context.page_factory.get_login_page(context.page)
    login_page.click_login_button()

Use page objects for interactions

def click_login_button(self):
    self.click_element(self.LOGIN_BUTTON)

Use Playwright assertions

expect(context.page).to_contain_text("Welcome")

🔍 Investigate Tracing

# Install Playwright trace viewer
npx playwright show-trace reports/traces/FILE_NAME.zip

# Or use the web interface
npx playwright show-trace --host 0.0.0.0 --port 8080 reports/traces/FILE_NAME.zip

📄 Log Files

Check log files for detailed error information:

reports/test.log - Detailed test execution logs
reports/workers/ - Parallel execution logs
reports/traces/ - Tracing reports

AI Selector Healing Logs

Monitor AI selector healing activities:

selector_map.json - Historical selector mappings and AI suggestions
selector_log.json - Detailed AI interaction logs with confidence scores
reports/screenshots/ai-*.png - Screenshots captured for AI analysis
Console output - Real-time AI healing notifications with emojis

Built with ❤️ using Python, Playwright, Behave, Allure, and AI-powered selector healing

Name		Name	Last commit message	Last commit date
Latest commit History 36 Commits
ai		ai
features		features
helpers		helpers
pages		pages
resources		resources
steps		steps
utils		utils
.gitignore		.gitignore
Personal-checks.txt		Personal-checks.txt
README.md		README.md
TASKS.md		TASKS.md
behave.ini		behave.ini
environment.py		environment.py
run_tests.py		run_tests.py
suggested-improvements.txt		suggested-improvements.txt

prashant1507/playwright-behave-allure-ai-framework

Folders and files

Latest commit

History

Repository files navigation