enhancement: simplify API #42

kikkomep · 2024-11-15T14:45:19Z

This PR simplifies the API for programmatic validation, enhances the code with additional documentation for clarity, and includes an initial example of programmatic usage in the README to help users get started.

(partially addresses issue #40)

…gs` interface

…l argument

…` object

…ocusNode`

simleo · 2024-11-18T11:29:27Z

README.md


+```bash
+RO-Crate is invalid!
+Detected issue of severity REQUIRED with check "ro-crate-1.1:root_entity_exists: The RO-Crate must contain a root entity.


The output sometimes changes when running the code multiple times. The list returned by result.get_issues contains only one element, and it may change in subsequent runs. I was expecting get_issues to return a list of all issues, instead it looks like only one is picked randomly.

probably has to do with whether fast failure is enabled

Could we switch to disabling fast failure by default? I've noticed that this is a pretty common point of confusion for new users (the CLI can also show different errors on different runs).

probably has to do with whether fast failure is enabled

Yes, it depends on the enabled fail-fast mode and, more specifically, on the underlying SHACL engine.

Could we switch to disabling fast failure by default? I've noticed that this is a pretty common point of confusion for new users (the CLI can also show different errors on different runs).

Sure, we can consider doing that, but perhaps in a separate PR.

ilveroluca · 2024-11-18T11:42:26Z

README.md

+    # Set the path to the RO-Crate root directory
+    rocrate_uri='../tests/data/crates/invalid/2_root_data_entity_metadata/missing_root_entity',
+    # Set the identifier of the RO-Crate profile to use for validation
+    profile_identifier='ro-crate-1.1',


wasn't this autodetected if unset?

Exactly, it was autodetected if unset.
I’ve added a comment about this in 29ca0ed

ilveroluca · 2024-11-18T11:43:30Z

README.md

+# Create an instance of `ValidationSettings` class to configure the validation
+settings = services.ValidationSettings(
+    # Set the path to the RO-Crate root directory
+    rocrate_uri='../tests/data/crates/invalid/2_root_data_entity_metadata/missing_root_entity',


I realize this is a real path, but for documentation it might be clearer to have something simpler (e.g., /path/to/rocrate)

fixed in bee0e3b

ilveroluca · 2024-11-18T11:45:17Z

README.md


+```bash
+RO-Crate is invalid!
+Detected issue of severity REQUIRED with check "ro-crate-1.1:root_entity_exists: The RO-Crate must contain a root entity.


probably has to do with whether fast failure is enabled

elichad · 2024-11-21T17:49:34Z

I haven't looked through all the code changes, but I've tried this out using the docs in this PR and it's a nice improvement!

I did find that when I got the example code to output all the errors (by disabling fast failure by setting abort_on_first=False in the ValidationSettings), some of the errors are duplicated, and some raw SHACL errors are surfaced:

$ python validate_crate.py 
RO-Crate is invalid!
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]

This is exacerbated if I choose a profile that inherits from the base profile, e.g. with workflow-ro-crate:

$ python validate_crate.py 
RO-Crate is invalid!
Detected issue of severity REQUIRED with check "workflow-ro-crate-1.0_4.1": Unable to check the existence of the main workflow file because the metadata file descriptor doesn't contain a `mainEntity`
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]

There might be some extra code needed to filter out duplicates as part of get_issues(). (Feel free to make this a separate issue if you prefer.)

… example

kikkomep · 2024-11-22T10:20:07Z

I haven't looked through all the code changes, but I've tried this out using the docs in this PR and it's a nice improvement!

I did find that when I got the example code to output all the errors (by disabling fast failure by setting abort_on_first=False in the ValidationSettings), some of the errors are duplicated, and some raw SHACL errors are surfaced:

$ python validate_crate.py 
RO-Crate is invalid!
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]

This is exacerbated if I choose a profile that inherits from the base profile, e.g. with workflow-ro-crate:

$ python validate_crate.py 
RO-Crate is invalid!
Detected issue of severity REQUIRED with check "workflow-ro-crate-1.0_4.1": Unable to check the existence of the main workflow file because the metadata file descriptor doesn't contain a `mainEntity`
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_5.2": The RO-Crate metadata file descriptor MUST have an `about` property referencing the Root Data Entity
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_9.1": The Root Data Entity MUST be a `Dataset` (as per `schema.org`)
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_10.1": The Root Data Entity URI MUST end with `/`
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//README.md>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow-test.yml>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//my-workflow.ga>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]
Detected issue of severity REQUIRED with check "ro-crate-1.1_13.1": Less than 1 values on <file://.//test-data/>->[ sh:inversePath <http://schema.org/hasPart> ]

There might be some extra code needed to filter out duplicates as part of get_issues(). (Feel free to make this a separate issue if you prefer.)

fixed in 9983589

…lass

kikkomep added 28 commits November 12, 2024 13:41

refactor(core): 🔥 remove internal settings from the `ValidationSettin…

46ed870

…gs` interface

refactor(core): ♻️ rename input parameter for the ro-crate uri

f62a8ae

fix(shacl): 🐛 update the condition when a check should be skipped

99a59e7

refactor(core): ♻️ update default value

f6eedab

refactor(core): ♻️ remove unused imports

0d1a52a

refactor(core): ♻️ init and validate URI on settings object

a7843bd

refactor(core): ♻️ link the context with the settings as an object

907a64f

test(core): ✅ update tests

9ea4e9c

refactor(core): ♻️ rename property to enable profile inheritance

f156f4c

refactor(core): ♻️ rename flag to disable reporting of inherited checks

c050141

refactor(core): ♻️ rename the flag to disable downloading a remote crate

6e6617f

refactor(core): 🔥 remove unused ontology_path setting

9f3a9ad

refactor(core): 🔥 remove cache timeout from validation settings

6001600

docs(core): 📝 add docstring of ValidationSettings class

97e3111

refactor(core): ♻️ add a docstring to the main validation classes

0d1cae1

refactor(core): ♻️ rename link property between result and RO-Crate URI

d9690c0

refactor(shacl): ♻️ mark internal methods

f4f7ea6

docs(core): 📝 add docstrings to the ValidationResult class

cfe353d

refactor(services): ♻️ the profile_identifier should be a positiona…

ce82884

…l argument

refactor(core): 💄 improve the readability of the identifier

bbb51ee

refactor(core): ♻️ update the representation of the `ValidationResult…

cc9d84b

…` object

chore(core): 🚨 fix flake8 issues

3eacae4

refactor(core): ♻️ update CheckIssue representation

cf67840

docs: 📝 add minimal example of programmatic usage

3bfbc2b

refactor(core): 🔥 remove obsolete code

934d7ca

refactor(core): ♻️ Use more descriptive names for resultPath and `f…

a33b229

…ocusNode`

docs(README): extend example

e176a91

docs: 💄 reformat

130b56e

ilveroluca requested review from ilveroluca and simleo November 15, 2024 16:08

simleo added 2 commits November 18, 2024 11:43

README: update list of supported profiles

d35b925

README: fix code snippet for programmatic validation

d94890b

simleo reviewed Nov 18, 2024

View reviewed changes

ilveroluca requested changes Nov 18, 2024

View reviewed changes

kikkomep added 3 commits November 19, 2024 12:32

refactor(services): 🔥 remove unused function argument

0aea7de

refactor(core): ♻️ rename some CheckIssue properties

b51d48d

refactor(core): ♻️ rename the parameter for ROCrate instantiation

d40181b

kikkomep added 2 commits November 22, 2024 09:37

docs(readme): 🐛 simplify path to crate in the programmatic validation…

bee0e3b

… example

fix(shacl): 🐛 avoid repeating errors

9983589

kikkomep added 2 commits November 22, 2024 11:33

docs(readme): 📝 add a comment on profile_identifier

29ca0ed

fix(shacl): ✨ skip redundant violation messages

bac3082

ilveroluca approved these changes Nov 25, 2024

View reviewed changes

refactor(core): ♻️ rename some properties of the ValidationResult c…

f252270

…lass

kikkomep merged commit a34d453 into crs4:develop Nov 25, 2024
2 checks passed

dnlbauer mentioned this pull request Nov 27, 2024

feat: swap default behaviour for fail fast #46

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

enhancement: simplify API #42

enhancement: simplify API #42

kikkomep commented Nov 15, 2024

simleo Nov 18, 2024

ilveroluca Nov 18, 2024

elichad Nov 21, 2024

kikkomep Nov 22, 2024

ilveroluca Nov 18, 2024

kikkomep Nov 22, 2024 •

edited

Loading

ilveroluca Nov 18, 2024

kikkomep Nov 22, 2024

ilveroluca Nov 18, 2024

elichad commented Nov 21, 2024

kikkomep commented Nov 22, 2024

enhancement: simplify API #42

enhancement: simplify API #42

Conversation

kikkomep commented Nov 15, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kikkomep Nov 22, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elichad commented Nov 21, 2024

kikkomep commented Nov 22, 2024

kikkomep Nov 22, 2024 •

edited

Loading