Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use indexing events to update the status of the release workflow #733

Open
thatbudakguy opened this issue Mar 11, 2024 · 0 comments
Open
Assignees
Labels
enhancement New feature or request

Comments

@thatbudakguy
Copy link
Member

thatbudakguy commented Mar 11, 2024

For discussion and previous work on the underlying problem, see sul-dlss/argo#3893.

As discussed at storytime on 3/11, we would like to try using the events currently published from the Searchworks and Earthworks indexers via dor-event-client to update the status of the release workflow. Specifically, several kinds of events are currently published:

  • indexing_success, for when an item was added to or updated in the Searchworks or Earthworks index
  • indexing_deleted, for when an item was removed from the Seachworks or Earthworks index
  • indexing_skipped, for when the indexer did not process an item (e.g. because of bad metadata)
  • indexing_errored, for when the indexer raised an error during indexing

These events are currently sent by the indexer after items are released via the releaseWF in Argo and processed via purl-fetcher's queue. We would like to use the indexing status of an item to update the release workflow, so that the workflow is only considered "completed" once a message is received from the indexer.

Edge cases to consider:

  • "managed PURLs", a 1:1 correspondence between an SDR object and a FOLIO record where the release is controlled via updating the record in FOLIO, and a link to the object is inserted into the MARC 856 field at index time
  • "free-range PURLs", a looser correspondence between a FOLIO record and an SDR object that is related to it in some way, where the 856 field is not managed automatically by the indexer (there are only a few hundred such objects)

Note that the Searchworks indexer does not currently report any events to SDR when FOLIO records are changed, although this could be implemented.

@thatbudakguy thatbudakguy added the enhancement New feature or request label Mar 11, 2024
@thatbudakguy thatbudakguy self-assigned this Mar 11, 2024
@thatbudakguy thatbudakguy changed the title Spike: use indexing events to update the status of the release workflow Use indexing events to update the status of the release workflow Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant