Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create commentary for recommended use practices #5

Open
roachspray opened this issue Sep 30, 2019 · 0 comments
Open

Create commentary for recommended use practices #5

roachspray opened this issue Sep 30, 2019 · 0 comments
Labels
big picture Related to a larger, overall question we have documentation Improvements or additions to documentation

Comments

@roachspray
Copy link
Contributor

Similar to the question of addressing the reality that this is a static suite of applications (#4 ), there is the question of what is the appropriate use of this suite. Any suite can (obviously?) be abused, so there should be some set of best practices that users of the suite would want to follow. Part of this list of practices should be inspired from documents like [1] [2] and others (there are some genetic/evol. algorithm comparison papers).

Things to be addressed should include:

  • Selection of a subset of apps
  • Number of trials
  • Duration of each trial
  • Statements on trial system's hardware, OS, etc
  • Acceptable seeds
  • ...I am sure this list is probably incomplete

[1] Klees, et al., "Evaluating Fuzz Testing", https://www.cs.umd.edu/~mwh/papers/fuzzeval.pdf
[2] Berger, et al., "A Checklist Manifesto for Empirical Evaluation: A Preemptive Strike Against a Replication Crisis in Computer Science", https://www.sigarch.org/a-checklist-manifesto-for-empirical-evaluation-a-preemptive-strike-against-a-replication-crisis-in-computer-science/

@roachspray roachspray added documentation Improvements or additions to documentation big picture Related to a larger, overall question we have labels Sep 30, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
big picture Related to a larger, overall question we have documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

1 participant