Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[24.1] Fix importing shared workflows with deeply nested subworkflows #19335

Merged

Conversation

mvdbeek
Copy link
Member

@mvdbeek mvdbeek commented Dec 17, 2024

We need to pass the user also to the workflow copy method when copying subworkflow steps. Adds a few bonus type annotations that made it easier to debug.

Fixes
https://sentry.galaxyproject.org/share/issue/454be6bbb98b43a1aa504309e1c6bc7b/::

NotNullViolation: null value in column "user_id" of relation "stored_workflow" violates not-null constraint
DETAIL:  Failing row contains (296863, 2024-12-16 17:04:34.329066, 2024-12-16 17:04:34.329067, null, null, TreeValGal bed to bigwig, f, f, null, f, null, t).

  File "sqlalchemy/engine/base.py", line 2116, in _exec_insertmany_context
    dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "user_id" of relation "stored_workflow" violates not-null constraint
DETAIL:  Failing row contains (296863, 2024-12-16 17:04:34.329066, 2024-12-16 17:04:34.329067, null, null, TreeValGal bed to bigwig, f, f, null, f, null, t).

[SQL: INSERT INTO stored_workflow (create_time, update_time, user_id, latest_workflow_id, name, deleted, hidden, importable, slug, from_path, published) SELECT p0::TIMESTAMP WITHOUT TIME ZONE, p1::TIMESTAMP WITHOUT TIME ZONE, p2::INTEGER, p3::INTEGER, p4:: ... 1138 characters truncated ... p9, p10, sen_counter) ORDER BY sen_counter RETURNING stored_workflow.id, stored_workflow.id AS id__1]
[parameters: {'from_path__0': None, 'user_id__0': 91803, 'name__0': 'TreeValGalBaseOneHaplotype', 'create_time__0': datetime.datetime(2024, 12, 16, 17, 4, 34, 329062), 'update_time__0': datetime.datetime(2024, 12, 16, 17, 4, 34, 329065), 'deleted__0': False, 'published__0': False, 'importable__0': False, 'latest_workflow_id__0': None, 'hidden...
  File "galaxy/web/framework/middleware/error.py", line 167, in __call__
    app_iter = self.application(environ, sr_checker)
  File "/cvmfs/main.galaxyproject.org/venv/lib/python3.11/site-packages/paste/httpexceptions.py", line 635, in __call__
    return self.application(environ, start_response)
  File "galaxy/web/framework/base.py", line 176, in __call__
    return self.handle_request(request_id, path_info, environ, start_response)
  File "galaxy/web/framework/base.py", line 271, in handle_request
    body = method(trans, **kwargs)
  File "galaxy/web/framework/decorators.py", line 94, in decorator
    return func(self, trans, *args, **kwargs)
  File "galaxy/webapps/galaxy/controllers/workflow.py", line 125, in imp
    self._import_shared_workflow(trans, stored)
  File "galaxy/webapps/base/controller.py", line 1161, in _import_shared_workflow
    session.commit()
  File "sqlalchemy/orm/scoping.py", line 597, in commit
    return self._proxied.commit()
  File "sqlalchemy/orm/session.py", line 2017, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
    # Copyright (C) 2005-2024 the SQLAlchemy authors and contributors
  File "sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "sqlalchemy/orm/session.py", line 1302, in commit
    self._prepare_impl()
  File "<string>", line 2, in _prepare_impl
    # Copyright (C) 2005-2024 the SQLAlchemy authors and contributors
  File "sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "sqlalchemy/orm/session.py", line 1277, in _prepare_impl
    self.session.flush()
  File "sqlalchemy/orm/session.py", line 4341, in flush
    self._flush(objects)
  File "sqlalchemy/orm/session.py", line 4476, in _flush
    with util.safe_reraise():
  File "sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "sqlalchemy/orm/session.py", line 4437, in _flush
    flush_context.execute()
  File "sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "sqlalchemy/orm/unitofwork.py", line 642, in execute
    util.preloaded.orm_persistence.save_obj(
  File "sqlalchemy/orm/persistence.py", line 93, in save_obj
    _emit_insert_statements(
  File "sqlalchemy/orm/persistence.py", line 1143, in _emit_insert_statements
    result = connection.execute(
  File "sqlalchemy/engine/base.py", line 1418, in execute
    return meth(
  File "sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
  File "sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
    ret = self._execute_context(
  File "sqlalchemy/engine/base.py", line 1844, in _execute_context
    return self._exec_insertmany_context(dialect, context)
  File "sqlalchemy/engine/base.py", line 2124, in _exec_insertmany_context
    self._handle_dbapi_exception(
  File "sqlalchemy/engine/base.py", line 2353, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "sqlalchemy/engine/base.py", line 2116, in _exec_insertmany_context
    dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)

(Please replace this header with a description of your pull request. Please include BOTH what you did and why you made the changes. The "why" may simply be citing a relevant Galaxy issue.)
(If fixing a bug, please add any relevant error or traceback)
(For UI components, it is recommended to include screenshots or screencasts)

How to test the changes?

(Select all options that apply)

  • I've included appropriate automated tests.
  • This is a refactoring of components with existing test coverage.
  • Instructions for manual testing are as follows:
    1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

  • I agree to license these and all my past contributions to the core galaxy codebase under the MIT license.

We need to pass the user also to the workflow copy method when copying
subworkflow steps. Adds a few bonus type annotations that made it easier
to debug.

Fixes
https://sentry.galaxyproject.org/share/issue/454be6bbb98b43a1aa504309e1c6bc7b/::

```
NotNullViolation: null value in column "user_id" of relation "stored_workflow" violates not-null constraint
DETAIL:  Failing row contains (296863, 2024-12-16 17:04:34.329066, 2024-12-16 17:04:34.329067, null, null, TreeValGal bed to bigwig, f, f, null, f, null, t).

  File "sqlalchemy/engine/base.py", line 2116, in _exec_insertmany_context
    dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
IntegrityError: (psycopg2.errors.NotNullViolation) null value in column "user_id" of relation "stored_workflow" violates not-null constraint
DETAIL:  Failing row contains (296863, 2024-12-16 17:04:34.329066, 2024-12-16 17:04:34.329067, null, null, TreeValGal bed to bigwig, f, f, null, f, null, t).

[SQL: INSERT INTO stored_workflow (create_time, update_time, user_id, latest_workflow_id, name, deleted, hidden, importable, slug, from_path, published) SELECT p0::TIMESTAMP WITHOUT TIME ZONE, p1::TIMESTAMP WITHOUT TIME ZONE, p2::INTEGER, p3::INTEGER, p4:: ... 1138 characters truncated ... p9, p10, sen_counter) ORDER BY sen_counter RETURNING stored_workflow.id, stored_workflow.id AS id__1]
[parameters: {'from_path__0': None, 'user_id__0': 91803, 'name__0': 'TreeValGalBaseOneHaplotype', 'create_time__0': datetime.datetime(2024, 12, 16, 17, 4, 34, 329062), 'update_time__0': datetime.datetime(2024, 12, 16, 17, 4, 34, 329065), 'deleted__0': False, 'published__0': False, 'importable__0': False, 'latest_workflow_id__0': None, 'hidden...
  File "galaxy/web/framework/middleware/error.py", line 167, in __call__
    app_iter = self.application(environ, sr_checker)
  File "/cvmfs/main.galaxyproject.org/venv/lib/python3.11/site-packages/paste/httpexceptions.py", line 635, in __call__
    return self.application(environ, start_response)
  File "galaxy/web/framework/base.py", line 176, in __call__
    return self.handle_request(request_id, path_info, environ, start_response)
  File "galaxy/web/framework/base.py", line 271, in handle_request
    body = method(trans, **kwargs)
  File "galaxy/web/framework/decorators.py", line 94, in decorator
    return func(self, trans, *args, **kwargs)
  File "galaxy/webapps/galaxy/controllers/workflow.py", line 125, in imp
    self._import_shared_workflow(trans, stored)
  File "galaxy/webapps/base/controller.py", line 1161, in _import_shared_workflow
    session.commit()
  File "sqlalchemy/orm/scoping.py", line 597, in commit
    return self._proxied.commit()
  File "sqlalchemy/orm/session.py", line 2017, in commit
    trans.commit(_to_root=True)
  File "<string>", line 2, in commit
    # Copyright (C) 2005-2024 the SQLAlchemy authors and contributors
  File "sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "sqlalchemy/orm/session.py", line 1302, in commit
    self._prepare_impl()
  File "<string>", line 2, in _prepare_impl
    # Copyright (C) 2005-2024 the SQLAlchemy authors and contributors
  File "sqlalchemy/orm/state_changes.py", line 139, in _go
    ret_value = fn(self, *arg, **kw)
  File "sqlalchemy/orm/session.py", line 1277, in _prepare_impl
    self.session.flush()
  File "sqlalchemy/orm/session.py", line 4341, in flush
    self._flush(objects)
  File "sqlalchemy/orm/session.py", line 4476, in _flush
    with util.safe_reraise():
  File "sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "sqlalchemy/orm/session.py", line 4437, in _flush
    flush_context.execute()
  File "sqlalchemy/orm/unitofwork.py", line 466, in execute
    rec.execute(self)
  File "sqlalchemy/orm/unitofwork.py", line 642, in execute
    util.preloaded.orm_persistence.save_obj(
  File "sqlalchemy/orm/persistence.py", line 93, in save_obj
    _emit_insert_statements(
  File "sqlalchemy/orm/persistence.py", line 1143, in _emit_insert_statements
    result = connection.execute(
  File "sqlalchemy/engine/base.py", line 1418, in execute
    return meth(
  File "sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
    return connection._execute_clauseelement(
  File "sqlalchemy/engine/base.py", line 1640, in _execute_clauseelement
    ret = self._execute_context(
  File "sqlalchemy/engine/base.py", line 1844, in _execute_context
    return self._exec_insertmany_context(dialect, context)
  File "sqlalchemy/engine/base.py", line 2124, in _exec_insertmany_context
    self._handle_dbapi_exception(
  File "sqlalchemy/engine/base.py", line 2353, in _handle_dbapi_exception
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "sqlalchemy/engine/base.py", line 2116, in _exec_insertmany_context
    dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 924, in do_execute
    cursor.execute(statement, parameters)
```
@mvdbeek mvdbeek force-pushed the nested_subworkflow_import_fix branch from 2ed36c9 to ea3626e Compare December 17, 2024 10:37
Copy link
Member

@nsoranzo nsoranzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@mvdbeek mvdbeek merged commit 3a195be into galaxyproject:release_24.1 Dec 18, 2024
44 of 50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants