WIP: 🎨 Enh/error handler 500

[pcrespov]

What do these changes do?

This PR enhances aiohttp api errors and specifically those resulting from an unhandled exception. These changes were motivated by experiencing how some unhandled exceptions on the server were displayed to the user with an illegible message. In the event of an error, it is desirable:

• to provide human readable messages to the user
  • With useful explanation of the problem
  • Only use "please contact support" as a last resource
• try to solve it automatically and/or provide options (e.g. resend confirmation email)
• if a server error, produce a comprehensive and traceable log

errors middleware

• 🎨 webserver's unexpected errors are returned as 500 with a friendly message and comprehensive logs
  • Human readable message (including OEC)
  • Log with OEC: traceability
  • Log with sufficient information to diagnose the issue:
• ♻️ improved error handling in aiohttp error's middleware
• Comprehensive testing of the concept in packages/service-library/tests/aiohttp/test_rest_middlewares.py

enveloped responses

• Preparations to deprecate envelope-middleware that enforces enveloped responses.
  • New policy: explicitly return envelope_json_response when applies. Why:
    • some cases e.g. pagination, the enveloping is done differently
    • other cases the don't even have envelope (e.g. redirections or no-content)
    • checking for envelopes in middleware became very error prone and costly
  • Adapted handlers in webserver
    • check front-end complies with webserver's openapi.json🚨

simplified error response

The idea is to allow the front-end to have at least a human-readable message to display (i.e. msg) and eventually extra information.

• Preparations to deprecate current error response payload model. Here some examples of validation errors for a request

{
	"error": {
		"loc": "path.project_uuid",
		"msg": "value is not a valid uuid",
		"type": "type_error.uuid",
	}
}

or for multiple errors

{
	"error": {
		"msg": "Invalid field/s 'body.x, body.z' in request",
		"details": [
			{
				"loc": "body.x",
				"msg": "field required",
				"type": "value_error.missing",
			},
			{
				"loc": "body.z",
				"msg": "field required",
				"type": "value_error.missing",
			},
		],
	}
 }       

where only msg is required. Another example with extra context variables is this

{
	"error": {
		"msg": "You have to activate your account via email, before you can login",
		"type": "activation_required",
		"ctx": {
		    "resend_email_url": "https://foo.io/resend?code=123456" 
		}
	}
}

• SEE more examples in test_requests_validation.py
• simplify error model used in the reponses? (sync with @odeimaiz )
• Check how front-end and api-server handlers error response models (sync with @odeimaiz )

status codes

• All http status codes in status module (i.e. from fastapi import status or from servicelib.aiohttp import status)
• serviceslib.status_utils: helpers free functions for status codes. Renamed helpers combining names and numbers as status does.
• In test_how_status_codes_map_to_aiohttp_exception_class we show that the lists of status codes differ in
  - status: the most complete list. Clear names and used by default.
  - http.HTTPStatus : python native. Provides phrases and short descriptions.
  - aiohttp.web_exceptions: Exception+Response classes for most of the status codes and extended to some more in servicelib.aiohttp.web_exceptions_extesion
• error codes (or OEC) are not status codes. Former are specific to osparc to identify an exception. The latter refer to standard http status codes.

Related issue/s

Follows up from series of PRs improving error handling repo-wide

• ♻️ Uniform naming of http status codes  #5323
• 🎨 Unified construction of error classes #5359
• API: Modify error and log responses in webserver rest API #443
• resolves Maintenance: enveloped_response decorator #3646

How to test

Driving tests

• packages/service-library/tests/aiohttp/test_rest_middlewares.py
• packages/service-library/tests/aiohttp/test_requests_validation.py

NOTES:

SEE aio-libs/aiohttp#2415

NOTE that aiohttp deprecated returning the HTTPExceptions and encourage them to be raised ...

aiohttp/web_protocol.py:451: DeprecationWarning: returning HTTPException object is deprecated (#2415) and will be removed, please raise the exception instead

and this needs to change.

return create_error_response(exc, http_error_cls=web.HTTPNotFound)

[codecov]

Codecov Report

Attention: Patch coverage is 34.09091% with 29 lines in your changes are missing coverage. Please review.

Project coverage is 76.3%. Comparing base (cafbf96) to head (392b578).
Report is 84 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #5487      +/-   ##
=========================================
- Coverage    84.5%   76.3%    -8.3%     
=========================================
  Files          10    1080    +1070     
  Lines         214   45119   +44905     
  Branches       25     551     +526     
=========================================
+ Hits          181   34453   +34272     
- Misses         23   10550   +10527     
- Partials       10     116     +106     

Flag	Coverage Δ
integrationtests	64.0% <34.0%> (?)
unittests	83.5% <ø> (-1.1%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
...rver/src/simcore_service_webserver/catalog/_api.py	39.1% <ø> (ø)
...er/src/simcore_service_webserver/exporter/utils.py	100.0% <100.0%> (ø)
.../simcore_service_webserver/login/_auth_handlers.py	64.5% <ø> (ø)
...rc/simcore_service_webserver/login/handlers_2fa.py	47.5% <ø> (ø)
...e_service_webserver/login/handlers_registration.py	47.4% <ø> (ø)
...erver/src/simcore_service_webserver/rest/plugin.py	96.0% <ø> (ø)
...ver/src/simcore_service_webserver/utils_aiohttp.py	57.1% <100.0%> (ø)
...rc/simcore_service_webserver/activity/_handlers.py	38.7% <50.0%> (ø)
...rc/simcore_service_webserver/clusters/_handlers.py	58.7% <0.0%> (ø)
...simcore_service_webserver/director_v2/_handlers.py	45.4% <66.6%> (ø)
... and 6 more

... and 1049 files with indirect coverage changes

[sonarqubecloud]

Quality Gate failed

Failed conditions
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[sonarqubecloud]

Quality Gate failed

Failed conditions
D Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint