Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/topic/robin/tests-from-file'
Browse files Browse the repository at this point in the history
* origin/topic/robin/tests-from-file:
  Add option to read lists of tests from file.
  • Loading branch information
rsmmr committed Jan 9, 2024
2 parents 905c8fe + 3a17cc1 commit f3d7e29
Show file tree
Hide file tree
Showing 6 changed files with 79 additions and 6 deletions.
14 changes: 14 additions & 0 deletions CHANGES
Original file line number Diff line number Diff line change
@@ -1,3 +1,17 @@
1.1-6 | 2024-01-09 09:16:00 +0100

* Add option to read lists of tests from file. (Robin Sommer)

```
-i FILE, --tests-file=FILE
Loads the list of tests to execute from a file. Each line in the
file is interpreted as the name of a test, or a group of tests, to
execute, just like the tests would be specified on the command
line. Empty lines and lines starting with ``#`` are ignored. (This
format is compatible with that of the ``btest`` `StateFile
<state_file_>`_.)
```

1.1-4 | 2023-10-04 11:30:31 +0200

* Simplify generation of temporary files (Benjamin Bannier, Corelight)
Expand Down
12 changes: 11 additions & 1 deletion README
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
..
.. Version number is filled in automatically.

.. |version| replace:: 1.1-4
.. |version| replace:: 1.1-6

==================================================
BTest - A Generic Driver for Powerful System Tests
Expand Down Expand Up @@ -400,6 +400,14 @@ and 1 otherwise. Exit code 1 can also result in case of other errors.
Retry any failed tests up to this many times to determine if
they are unstable.

-i FILE, --tests-file=FILE
Loads the list of tests to execute from a file. Each line in the
file is interpreted as the name of a test, or a group of tests, to
execute, just like the tests would be specified on the command
line. Empty lines and lines starting with ``#`` are ignored. (This
format is compatible with that of the ``btest`` `StateFile
<state_file_>`_.)

.. _configuration file: configuration_
.. _configuration:

Expand Down Expand Up @@ -558,6 +566,8 @@ configuration file:
range. The default range is "1024-65535".

``StateFile``
.. _state_file:

The name of the state file to record the names of failing tests. Default is
``.btest.failed.dat``.

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.1-4
1.1-6
29 changes: 25 additions & 4 deletions btest
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ else:
import multiprocessing.managers as mp_managers
import multiprocessing.sharedctypes as mp_sharedctypes

VERSION = "1.1-4" # Automatically filled in.
VERSION = "1.1-6" # Automatically filled in.

Name = "btest"
Config = None
Expand Down Expand Up @@ -193,12 +193,12 @@ def getOption(key, default, section="btest"):
reBackticks = re.compile(r"`(([^`]|`)*)`")


def readStateFile():
def readTestsFile(file):
try:
# Read state file.
tests = []

for line in open(StateFile):
for line in open(file, encoding="utf-8"):
line = line.strip()
if not line or line.startswith("#"):
continue
Expand Down Expand Up @@ -2937,6 +2937,15 @@ def parse_options():
"Can be specified multiple times."
),
)
optparser.add_option(
"-i",
"--tests-file",
action="store",
type="string",
dest="tests_file",
default="",
help=("Load list of tests to execute from file."),
)

optparser.set_defaults(mode="TEST")

Expand Down Expand Up @@ -3201,11 +3210,17 @@ if __name__ == "__main__":
Config.configured_tests = findTests(testdirs, True)

if args:
if Options.tests_file:
error("cannot specify tests both on command line and with --tests-file")

tests = findTests(args)

else:
if Options.rerun:
(success, tests) = readStateFile()
if Options.tests_file:
error("cannot specify both --rerun and --tests-file")

(success, tests) = readTestsFile(StateFile)

if success:
if not tests:
Expand All @@ -3216,6 +3231,12 @@ if __name__ == "__main__":
warning("cannot read state file, executing all tests")
tests = Config.configured_tests

elif Options.tests_file:
(success, tests) = readTestsFile(Options.tests_file)

if not success:
warning("cannot read file with tests: %s" % Options.tests_file)

else:
tests = Config.configured_tests

Expand Down
8 changes: 8 additions & 0 deletions testing/Baseline/tests.tests-file/output
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
t1 ... ok
t2 ... failed
t3 ... ok
1 of 3 tests failed
t1 ... ok
t3 ... ok
all 2 tests successful
20 changes: 20 additions & 0 deletions testing/tests/tests-file.test
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# %TEST-EXEC-FAIL: btest t1 t2 t3 >>output 2>&1
# %TEST-EXEC: btest -i my-tests >>output 2>&1
# %TEST-EXEC: btest-diff output

%TEST-START-FILE t1
@TEST-EXEC: exit 0
%TEST-END-FILE

%TEST-START-FILE t2
@TEST-EXEC: exit 1
%TEST-END-FILE

%TEST-START-FILE t3
@TEST-EXEC: exit 0
%TEST-END-FILE

%TEST-START-FILE my-tests
t1
t3
%TEST-END-FILE

0 comments on commit f3d7e29

Please sign in to comment.