fixsuite-r48 fails with latest randoop

[markro49]

I pulled and build Randoop this morning (5/28).

~/grt-testing/scripts $ ./mutation.sh -o fs1.csv  -t 60 -f BLOODHOUND fixsuite-r48 |& tee fs1.log

Running mutation test on fixsuite-r48

TIME_LIMIT: 60 seconds

Checked out include-major.
/homes/gws/markro/grt-testing/scripts
Using Randoop to generate tests.

Using BLOODHOUND

Randoop for Java version "4.3.3, local changes, branch master, commit 16d92d4, 2025-05-27".

Will try to generate tests for 25 classes.
PUBLIC MEMBERS=740
Explorer = ForwardGenerator(steps: 0, null steps: 0, num_sequences_generated: 0;
    allSequences: 0, regresson seqs: 0, error seqs: 0=0=0, invalid seqs: 0, subsumed_sequences: 0, num_failed_output_test: 0;
    sideEffectFreeMethods: 1115, runtimePrimitivesSeen: 38)

Progress update: steps=1, test inputs generated=0, failing inputs=0      (2025-05-28T18:02:18.609Z     461M used)log4j:WARN No appenders could be found for logger (org.fixsuite.message.Library).
log4j:WARN Please initialize the log4j system properly.

[markro49]

Fails the same way with ORIENTEERING

[markro49]

as of 12:48 6/19/2025 still fails same way for me. Note the last line of log file: "log4j:WARN Please initialize the log4j system properly." Perhaps there is something missing in the setup instructions about log4j?

[776styjsu]

@mernst

Description

When running the mutation script against fixsuite-r48, FixSuite’s LibrarySuite fails during GUI initialization and Randoop exits prematurely with an NPE.

Environment Setup

1. Clone the repo and switch branch

   git clone git@github.com:776styjsu/grt-testing.git
   cd grt-testing
   git checkout fixsuite-fix

2. Install prerequisites (per scripts/mutation-prerequisites.md):

   make -C scripts
   make -C scripts randoop-from-source
   ./scripts/get-all-subject-src.sh

3. Ensure Java runs in non-headless mode by editing scripts/mutation.sh (line 393):

   -Djava.awt.headless=true
   +Djava.awt.headless=false

4. Enable Additional Logging (optional)

   cd subject-programs/src/fixsuite-r48/
   git checkout include-major
   # Modify log4j.properties to set the logger
   

Reproduction Steps

1. From the grt-testing repo root:

   ./scripts/mutation.sh -t 30 -o coverage.csv fixsuite-r48

Behavior

The script terminates with:

ERROR org.fixsuite.message.LibrarySuite - An error occurred while initializing the UI: null
java.lang.NullPointerException
    at org.fixsuite.message.view.ListView$MessageListModel.load(Unknown Source)
    at org.fixsuite.message.view.ListView.<init>(Unknown Source)
    at org.fixsuite.message.LibrarySuite.initGUI(Unknown Source)
    at org.fixsuite.message.LibrarySuite.access$600(Unknown Source)
    at org.fixsuite.message.LibrarySuite$LibraryLoader.run(Unknown Source)
    at java.lang.Thread.run(Thread.java:750)

Cause

In org.fixsuite.message.view.ListView$MessageListModel (line 336), the internal dictionary field is null. Since load(...) is private, Randoop cannot specify a pre-condition to prevent the NPE.

Temporary Workaround

Running in headless mode (-Djava.awt.headless=true) bypasses the GUI initialization and thus the NPE. This may limit how many classes Randoop can generate tests for.

Questions

1. Is there any alternative way to assert a non-null DictionaryInfo before load(...) runs?

2. Is it acceptable that we modify the source code of fixsuite-r48 to expose load(...) to allow a pre-condition specification?

3. If it becomes necessary to exclude some classes or methods from testing to avoid this issue, which classes or methods could be safely omitted with minimal impact?

[varuniy]

BTW running in headless mode significantly reduces the number of classes included -- nearly half of FixSuite's classes are excluded -- which obviously results in a substantial impact.