Run the upstream test suite in CI (fixes #8)

[oSoMoN]

The script takes a shortcut to fetch only the test suite from the upstream repository and carefully avoids running the autotools machinery which is time-consuming and resource-intensive, and doesn't offer the option to not build the upstream binaries.
As a consequence, the environment in which the tests are run might not match exactly that used when the upstream tests are run through the autotools. Whether tests are failing because of an incorrect environment remains to be investigated.

Unsurprisingly a number of tests are failing because of missing options in the current implementation or discrepancy in the output. These will need to be implemented/fixed.

[sylvestre]

The job should not fail.
And we should generate a json at the end like the coreutils but it can be done in another pr.

Please let me know when it is ready for review

[oSoMoN]

The job should not fail. And we should generate a json at the end like the coreutils but it can be done in another pr.

I tweaked the workflow to not fail when run in CI. I'll look into generating that JSON data as soon as I get a chance.

Please let me know when it is ready for review

Almost there, I would like to allow run-upstream-testsuite.sh to run selected tests only, and to report results more verbosely to ease investigation of failures. I'm converting this PR to draft until it's ready for review.

[sylvestre]

Do you think it would be possible to show the errors in the log ? thanks

[oSoMoN]

I believe this is now ready for review.

Do you think it would be possible to show the errors in the log ? thanks

I added a script that prints out details about every test (including the contents of stdout and stderr) in a markdown format, convenient for filing issues to track tests with an unexpected result. When rendered it looks like this:

test: excess-slash

result: FAIL
url: https://git.savannah.gnu.org/cgit/diffutils.git/tree/tests/excess-slash?id=6c88db1f7bfc25b1480d9876dac25024aa688544

stdout

diff -u /dev/null err
--- /dev/null	1970-01-01
+++ err	1970-01-01
+Error: "Unknown option: r"
--- expected-out	
+++ out	
@@ -1 +0,0 @@
-Only in b/f: g
Not removing temporary directory /tmp/tmp.3jNgxNWI94/diffutils/tests/gt-excess-slash.YClC

[sylvestre]

excellent, thanks