WIP: 🐛 Clean removal of output files

[giancarloromeo]

What do these changes do?

TL;DR: Removing (output) files referenced by Nodes, leaves the system in an inconsistent state.

Currently, the WebServer is the 1st responder and forwards the deletion requests to the Storage service, which deletes physically the file(s).
The Projects/Nodes are not updated accordingly and director v2 is not informed about the changes.

Related issue/s

• Resolves: Removing a linked output file in a service, makes the next service unrunnable without clear reason #6963

How to test

Dev-ops checklist

• No ENV changes or I properly updated ENV (read the instruction)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.49%. Comparing base (493488c) to head (abc4ce0).

❗ There is a different number of reports uploaded between BASE (493488c) and HEAD (abc4ce0). Click for more details.

HEAD has 26 uploads less than BASE

Flag	BASE (493488c)	HEAD (abc4ce0)
unittests	29	3

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #6986      +/-   ##
==========================================
- Coverage   85.61%   77.49%   -8.13%     
==========================================
  Files        1621      649     -972     
  Lines       63993    31679   -32314     
  Branches     2035      262    -1773     
==========================================
- Hits        54789    24550   -30239     
+ Misses       8871     7069    -1802     
+ Partials      333       60     -273     

Flag	Coverage Δ
integrationtests	64.86% <ø> (+0.27%)	⬆️
unittests	79.91% <ø> (-4.49%)	⬇️

Components	Coverage Δ
api	∅ <ø> (∅)
pkg_aws_library	∅ <ø> (∅)
pkg_dask_task_models_library	∅ <ø> (∅)
pkg_models_library	∅ <ø> (∅)
pkg_notifications_library	∅ <ø> (∅)
pkg_postgres_database	∅ <ø> (∅)
pkg_service_integration	∅ <ø> (∅)
pkg_service_library	∅ <ø> (∅)
pkg_settings_library	∅ <ø> (∅)
pkg_simcore_sdk	77.37% <ø> (-8.02%)	⬇️
agent	∅ <ø> (∅)
api_server	∅ <ø> (∅)
autoscaling	∅ <ø> (∅)
catalog	∅ <ø> (∅)
clusters_keeper	∅ <ø> (∅)
dask_sidecar	∅ <ø> (∅)
datcore_adapter	∅ <ø> (∅)
director	∅ <ø> (∅)
director_v2	78.75% <ø> (-12.67%)	⬇️
dynamic_scheduler	∅ <ø> (∅)
dynamic_sidecar	59.86% <ø> (-29.89%)	⬇️
efs_guardian	∅ <ø> (∅)
invitations	∅ <ø> (∅)
osparc_gateway_server	∅ <ø> (∅)
payments	∅ <ø> (∅)
resource_usage_tracker	∅ <ø> (∅)
storage	∅ <ø> (∅)
webclient	∅ <ø> (∅)
webserver	79.88% <ø> (-0.02%)	⬇️

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 493488c...abc4ce0. Read the comment docs.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[giancarloromeo]

The issue is that we have 2 separate services (storage and webserver/projects) that are not consistent. Whenever a request of file deletion is received from the UI, the file is deleted from the storage but the project/s referencing it is/are not updated.