Skip to content

feat(FR-1329): add a github action for regression test #4428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat(FR-1329): add a github action for regression test #4428

wants to merge 1 commit into from
+150 −8

Conversation

@nowgnuesLee
Copy link
Contributor

@nowgnuesLee nowgnuesLee commented Oct 21, 2025 •
edited
Loading

Add E2E Testing with Playwright for CI

This PR adds end-to-end testing capabilities using Playwright that run automatically on pull requests. The implementation:

  • Creates a new GitHub workflow for running Playwright tests against different LTS versions
  • Enhances the login utility to support different sign-in modes
  • Configures Playwright to work properly in CI environments
  • Adds a helper function to switch between IAM and ID sign-in modes
  • Updates the login component with a data-testid attribute for better test targeting
  • Adjusts timeout and reporter settings for CI compatibility

Checklist:

  • Documentation
  • Minium required manager version
  • Specific setting for review (eg., KB link, endpoint or how to setup)
  • Minimum requirements to check during review
  • Test case(s) to demonstrate the difference of before/after

@github-actions github-actions bot added the size:L 100~500 LoC label Oct 21, 2025
@nowgnuesLee Graphite App
Copy link
Contributor Author

nowgnuesLee commented Oct 21, 2025 •
edited
Loading

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • flow:merge-queue - adds this PR to the back of the merge queue
  • flow:hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 21, 2025
View reviewed changes
@nowgnuesLee nowgnuesLee force-pushed the feat/action-for-regression-test branch 2 times, most recently from ceb5628 to e6ed8de Compare October 21, 2025 11:13
github-advanced-security[bot]
github-advanced-security bot found potential problems Oct 21, 2025
View reviewed changes
@nowgnuesLee nowgnuesLee force-pushed the feat/action-for-regression-test branch 6 times, most recently from c79e0fd to fc952ad Compare October 23, 2025 04:06
@nowgnuesLee nowgnuesLee force-pushed the feat/action-for-regression-test branch 2 times, most recently from 2348987 to 6b456de Compare November 17, 2025 08:30
@nowgnuesLee nowgnuesLee marked this pull request as ready for review November 17, 2025 09:03
@nowgnuesLee nowgnuesLee requested review from Copilot, ironAiken2 and yomybaby and removed request for Copilot November 17, 2025 09:03
Copilot finished reviewing on behalf of nowgnuesLee November 17, 2025 09:06
@nowgnuesLee nowgnuesLee force-pushed the feat/action-for-regression-test branch from 6b456de to 080c2e5 Compare November 25, 2025 07:29
Copilot AI review requested due to automatic review settings November 25, 2025 07:29
Copilot AI reviewed Nov 25, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds end-to-end testing with Playwright to run automatically on pull requests against different LTS versions of the backend API. The implementation configures CI workflows, enhances login utilities to support IAM/ID sign-in mode switching, and adjusts Playwright configuration for CI environments.

Key Changes:

  • New GitHub Actions workflow for running Playwright tests against multiple LTS versions
  • Enhanced login utility with sign-in mode switching and conditional endpoint handling
  • CI-optimized Playwright configuration with adjusted timeouts and reporters

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
.github/workflows/e2e-test.yml Implements GitHub Actions workflow for E2E testing with matrix strategy for LTS versions
e2e/utils/test-util.ts Adds sign-in mode switcher function and enhances login utility with conditional endpoint input handling
playwright.config.ts Configures CI-specific timeout settings and reporter options
src/components/backend-ai-login.ts Adds data-testid attribute for test targeting
tsconfig.json Includes Playwright config in TypeScript compilation

e2e/utils/test-util.ts
export const webServerEndpoint = 'http://127.0.0.1:8090';
export const visualRegressionWebserverEndpoint = 'http://10.122.10.216:8090';

export const changeSigninInMode = async (page: Page, mode: 'IAM' | 'ID') => {
Copy link

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected spelling of 'changeSigninInMode' to 'changeSignInMode' (capitalized 'I' in 'In').

Copilot uses AI. Check for mistakes.
e2e/utils/test-util.ts
endpoint: string,
) {
await page.goto(webuiEndpoint);
await changeSigninInMode(page, 'ID');
Copy link

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Function call should use corrected name 'changeSignInMode' to match the function definition.

Copilot uses AI. Check for mistakes.
playwright.config.ts
retries: process.env.CI ? 2 : 0,
/* Opt out of parallel tests on CI. */
workers: process.env.CI ? 1 : undefined,
// workers: process.env.CI ? 4 : undefined,
Copy link

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented-out code should be removed rather than left in the codebase. If parallel workers are intentionally disabled, consider adding a comment explaining why.

Suggested change
// workers: process.env.CI ? 4 : undefined,

Copilot uses AI. Check for mistakes.
playwright.config.ts
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
trace: "on-first-retry",
},
timeout: process.env.CI ? 10 * 1000 : undefined,
Copy link

Copilot AI Nov 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The 10-second timeout for CI might be too short for E2E tests. Consider documenting why this specific timeout was chosen or using a named constant to make the value more maintainable.

Copilot uses AI. Check for mistakes.
ironAiken2
ironAiken2 requested changes Dec 3, 2025
View reviewed changes
Copy link
Contributor

@ironAiken2 ironAiken2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check the copilot review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@ironAiken2 ironAiken2 ironAiken2 requested changes

@yomybaby yomybaby Awaiting requested review from yomybaby

Requested changes must be addressed to merge this pull request.

Assignees

@nowgnuesLee nowgnuesLee

Labels

size:L 100~500 LoC

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@nowgnuesLee @ironAiken2