TestGenie is an external graphical IntelliJ plugin that integrates EvoSuite into the IDE. EvoSuite is a tool that automatically generates test cases with assertions for classes written in Java code. TestGenie makes this much easier, as it provides an intuitive modern interface for EvoSuite – so, no more CLI.
Please keep in mind that tests generated by TestGenie are meant to augment your existing test suites. They are not meant to replace writing tests manually.
- EvoSuite is installed with the plugin
- Automatic test generation for classes
- Automatic test generation for methods
- Automatic test generation for lines
- Coverage table
- Coverage visualisation for generated test suite
- Killed mutants visualisation
- Caching tests
- Test validation
- Accessibility features
- Telemetry
TestGenie uses EvoSuite to automatically generate tests for Java classes. Two clicks are required from the user for the tests to be generated.
TestGenie uses EvoSuite to automatically generate tests for Java methods. Two clicks are required from the user for the tests to be generated.
TestGenie uses EvoSuite to automatically generate tests for Java code lines. Two clicks are required from the user for the tests to be generated.
Basic statistics for the test suite in form of a table.
TestGenie offers an option to highlight lines which the generated test suite covers. Clicking the gutter icon (green rectangle) to the left of the lines allows the user to see which tests cover the line. Clicking a name of a test highlights the according test in the "Generated Tests" tab to make it easy to spot.
TestGenie offers an option to see which mutants were killed and by which tests. Clicking the gutter icon (green rectangle) to the left of the lines allows the user to see which tests cover the line. Clicking a name of a mutant highlights all the tests in the "Generated Tests" tab which cover it.
Due to its nature, generating tests with EvoSuite takes time. TestGenie takes steps to address this as much as technically feasible by caching all generated test cases that the user has decided not to apply immediately. This then allows TestGenie to instantly show those tests later, when the user decides to generate tests for a method or class that those unapplied tests also covered.
To assure that the cached tests are still valid, we have static and dynamic validation which are run before showing any cached tests.
Features which make the plugin more user-friendly. For instance, a color picker for coverage visualisation.
Opt-in non-intrusive data collection to improve EvoSuite in the future.
-
Using IDE built-in plugin system:
Settings/Preferences > Plugins > Marketplace > Search for "TestGenie" > Install Plugin
-
Manually:
Download the latest release and install it manually using Settings/Preferences > Plugins > ⚙️ > Install plugin from disk...
If you are running the plugin for the first time, checkout the First time configuration section.
To generate a test for a class, right-click (with mouse) anywhere within the class you want to test or right-click the class name itself (note that when using multiple cursors only the last one will count). Under the "TestGenie" option, select "Generate Tests for Class [...]" option:
EvoSuite going to run in the background and generate tests. While EvoSuite is running, a progress bar in the bottom right-hand corner of the IDE:
Once the tests are generated, they can be accessed in the "TestGenie" tool window (under "Generated Tests" tab), accessible via the stripe button on the right-hand side of the IDE:
You can examine the created tests, select those you wish to be added to your test class (via checkboxes). The tests can also be edited inside their mini-editors. Hitting the "Apply to test suite" button will add the selected tests to a test class of your choice.
Additionally, the top row of the tool window has buttons for selecting all tests, deselecting all tests, removing (deleting) all tests, and validating them.. The user also has an overview of how many tests they currently have selected.
To generate a test for a method, right-click (with mouse) anywhere within the method you want to test or right-click the method name itself (note that when using multiple cursors only the last one will count). Under the "TestGenie" option, select "Generate Tests for Method [...]" option:
EvoSuite going to run in the background and generate tests. While EvoSuite is running, a progress bar in the bottom right-hand corner of the IDE:
Once the tests are generated, they can be accessed in the "TestGenie" tool window (under "Generated Tests" tab), accessible via the stripe button on the righthand-side of the IDE:
You can examine the created tests, select those you wish to be added to your test class (via checkboxes). The tests can also be edited inside their mini-editors. Hitting the "Apply to test suite" button will add the selected tests to a test class of your choice.
Additionally, the top row of the tool window has buttons for selecting all tests, deselecting all tests, removing (deleting) all tests, and validating them.. The user also has an overview of how many tests they currently have selected.
To generate a test for a method, right-click (with mouse) anywhere within the line you want. Note that the line has to contain a statement (e.g. you will not have the option on lines with only method declarations). Under the "TestGenie" option, select "Generate Tests for Line [...]" option:
EvoSuite going to run in the background and generate tests. While EvoSuite is running, a progress bar in the bottom right-hand corner of the IDE:
Once the tests are generated, they can be accessed in the "TestGenie" tool window (under "Generated Tests" tab), accessible via the stripe button on the righthand-side of the IDE:
You can examine the created tests, select those you wish to be added to your test class (via checkboxes). The tests can also be edited inside their mini-editors. Hitting the "Apply to test suite" button will add the selected tests to a test class of your choice.
Additionally, the top row of the tool window has buttons for selecting all tests, deselecting all tests, removing (deleting) all tests, and validating them.. The user also has an overview of how many tests they currently have selected.
Once a test suite is generated, basic statistics about it can be seen in the tool window, coverage tab. The statistics include line coverage, branch coverage, weak mutation coverage. The table adjusts dynamically - it only calculates the statistics for the selected tests in the test suite.
For coverage visualisation to work, you must have it turned on. The setting is available in the Quick Access tab.
Once test are generated, the lines which are covered by the tests will be highlighted (default color: green). The gutter next to the lines will have a green rectangle as well. If the rectangle is clicked, a popup will show the names of the tests which cover the selected line. If any of the test names are clicked, the corresponding test in the toolwindow will be highlighted with the same accent color. The effect lasts 10 seconds. Coverage visualisation adjusts dynamically - it only shows results for the tests that are selected in the TestGenie tab.
For reference, without visualisation the window would look like this:
For mutation visualisation to work, you must have coverage visualisation enabled. You must also have the criterion Mutation coverage turned on. The setting is available in the Settings > Tools > TestGenie > EvoSuite tab.
Once tests are generated, the same gutter from Coverage Visualisation can be used to show which mutants are covered by the current test suite and which ones are not. For mutants covered, the mutant can be clicked. Clicking the mutant will highlight all the tests that kill the mutant. Mutation visualisation adjusts dynamically - it only shows results for the tests that are selected in the TestGenie tab.
When a user initially generates tests for a class, this will take some time, as the EvoSuite backend needs to be invoked. Then, if a user generates tests for a single method in this class, Test-Genie will look in its cache and find the tests that were previously generated for the entire class, and will instantly display them to the user rather than invoking EvoSuite again.
Before displaying cached tests, they are (in)validated statically and dynamically.
Tests which are no longer representative of the current state of the code under test are dropped from the cache, as they are no longer valid. We have 2 types of cache validation - static and dynamic validation. Static validation removes tests from cache by analysing the structure of the methods inside a class. If a method is modified, the tests covering it would be invalidated. Furthermore, it ignores whitespace changes, formatting inside a method and also reordering methods inside a class. Dynamic invalidation comes after static invalidation and verifies that all tests in the cache are still valid by running each of them. Failing tests are removed from the cache.
On the tool window, in the top corner there exist two buttons: Validate Tests and Toggle Jacoco Report (greyed out). To validate cached tests, the user can press Validate Tests. Validation is going to happen in the background.
Once validated, Toggle Jacoco Report becomes clickable. If clicked, a jacoco table with the updated coverage (so, excluding deleted tests) will appear.
- The plugin supports changing the color for coverage visualisation and killed mutants visualisation (one setting for both). To change the color, go to Settings > Tools > TestGenie and use the color picker under
Accessibility settings:
- The plugin has been designed with translation in mind. The vast majority of the plugins labels, tooltips, messages, etc. is stored in .property files. For more information on translation, refer to the contributing readme.
One of the biggest future plans of our client is to leverage the data that is gathered by TestGenie’s telemetry. This will help them with future research, including the development of an interactive way of using EvoSuite. The general idea behind this feature is to learn from the stored user corrections in order to improve test generation.
To opt into telemetry, go to Settings > Tools > TestGenie and tick the Enable telemetry checkbox. If you want, change the directory where telemetry is stored.
The plugin is configured mainly through the Settings menu. The plugin settings can be found under Settings > Tools > TestGenie. Here, the user is able to select options for the plugin:
Before running the plugin for the first time, we highly recommend going to the Environment settings section of TestGenie settings. The settings include java 11 path, compilation path (path to compiled code), compilation command. All commands have defaults. However, we recommend especially that you check compilation command. For this command the user requires maven, gradle or any other builder program which can be accessed via command. Leaving this field with a faulty value may cause unintended behaviour.
Some parameters for tweaking EvoSuite are used so often, that going to the settings menu gets annoying. That why these parameters were added (exclusively) to the Quick Access Panel of the TestGenie tool window (Parameters tab). These are settings so common/useful that they deserved their own spot:
❗ Pro tip: don't forget to hit the "save" button at the bottom. ❗
The settings submenu Settings > Tools > TestGenie > EvoSuite allows the user to tweak EvoSuite parameters to their liking. EvoSuite has hundreds of parameters, not all can be packed in a settings menu. However, the most commonly used and rational settings were added here:
The plugin is Open-Source and publicly hosted on github. Anyone can look into the code and suggest changes. You can find the plugin page here.
Plugin based on the IntelliJ Platform Plugin Template.