You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
I would like to have the ability to track file provenance for each files used (input) or created (output) by an App. The provenance information should include:
File name
Creation date
File size
What App created it (if it doesn't already exist)
What arguments were given to the App
What environment was the App running in
What other Apps used the file
Describe the solution you'd like
The system would need to be able to track files (already does), and capture information about them when they are created, or used for the first time, then track their usage through the rest of the workflow. Ideally, this should require minimal changes to existing workflows. Using the existing monitoring framework is a good candidate as it already has the ability to log information to a database.
Describe alternatives you've considered
The only alternatives I have come up with are manually tracking the input and output files (not too bad for a small workflow, but very laborious, if not impossible for large workflows), and adding code in each App to monitor files created (but this would require also creating infrastructure that is alredy provided by the monitoring framework).
Additional context
It would be nice to be able to easily access the provenance information, perhaps through the parsl-visualizer.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
I would like to have the ability to track file provenance for each files used (input) or created (output) by an App. The provenance information should include:
Describe the solution you'd like
The system would need to be able to track files (already does), and capture information about them when they are created, or used for the first time, then track their usage through the rest of the workflow. Ideally, this should require minimal changes to existing workflows. Using the existing monitoring framework is a good candidate as it already has the ability to log information to a database.
Describe alternatives you've considered
The only alternatives I have come up with are manually tracking the input and output files (not too bad for a small workflow, but very laborious, if not impossible for large workflows), and adding code in each App to monitor files created (but this would require also creating infrastructure that is alredy provided by the monitoring framework).
Additional context
It would be nice to be able to easily access the provenance information, perhaps through the parsl-visualizer.
The text was updated successfully, but these errors were encountered: