This package provides a script, classroom-tool which is designed to work with
GitHub Classroom to automate some of the tasks
an instructor might have in marking work submitted on that platform.
The assumed workflow is that students undertake a GitHub classroom assignment, which they push to GitHub by a given date and time. The instructor(s) mark the work on a separate, private, GitHub repository. This repository contains one pull request per student, which enables the instructor(s) to mark the work by commenting on the pull request. The instructors' comments are returned to the students by creating PDFs of the annotated pull requests.
Install the package using:
$ pip install classroom-toolclassroom-tool will need to access GitHub in order to work, and to do so it
will need to be able to authenticate to GitHub. You will therefore need to set
up a GitHub personal access
token
with repo permissions. Store the PAT in the GITHUB_PAT environment variable
so that classroom-tool can access it.
When configuring the assignment on GitHub Classroom, ensure that you select
Enable feedack pull requests. classroom-tool will need these as the base
for the marking pull requests in the marking repository.
Configuration information which is constant for a particular marking exercise needs to be provided in a configuration file in ConfigParser format. A simple example follows.
[students]
roster = classroom_roster.csv
[github]
organization = my-org
basename = exam-2021
marking-repo = marking-exam-2021
[assignment]
deadline = 2021-05-26T12:00:00+01:00
This section contains information about the students taking the exercise.
roster is a .csv (comma-separated values) file with at least the following
columns. These are present in the roster you can download from GitHub classroom:
identifier: The identifier of the student at your institution, such as a
student number or username.
github_username: The GitHub username of the student.
Any other columns will be preserved in output. The following optional column has special meaning:
extra_time: The additional time allowed to a student in minutes (for example
as a result of a disability). This can be blank for students with no additional
time.
This section contains the (non-sensitive) GitHub information about the assignment.
organization: The name of the GitHub organization containing the classroom.
basename: The GitHub classroom prefix for the current assignment.
marking-repo: The name of the repository in which the marking branches and
pul requests will be created. This is in the same organization. The marking
repository should be an initially empty, private repository. Its name
should not start with basename.
delay: The number of seconds to wait between creating pull requests. Set or
increase this if you find you are hitting GitHub secondary rate limits when
running with the --pull-requests option. Defaults to 10 seconds.
Information specific to this assignment.
deadline: The submission deadline in iso 8601 time and data format.
Note that GitHub does not record push times, so this is the timestamp on the last commit which will be accepted.
classroom-tool assumes a sequential processing of the submissions, with
potential manual intervention at each stage to cater for anomolous inputs or
circumstances.
The basic usage is:
classroom-tool --config-file CONFIG_FILE
with CONFIG_FILE replaced by the name of the configuration file. By itself,
this will do nothing. Actual processing is achieved by adding the options for
one or more of the following, stages. A help message is available by passing
--help.
Passing --fetch will fetch all the existant student branches from GitHub.
Passing --create-branches will create local branches with name
identifier-main (or identifier-master) and identifier-feedback. Here,
identifier is taken from the class roster in the configuration file, as
opposed to the GitHub username. If the identifier is anonymous (such as a
student number), this forms the first stage in anonymising the submissions.
Passing --impose-deadline will create local branches with names of the form
identifier-mark. These branches will point to the last commit before the
deadline specified in the configuration file. In the current version of
classroom-tool, extra time is not automatically accounted for at this stage:
you will need to manually move the branch pointer in these cases. For example
if student 123456 had additional time and you wish to mark their very last
commit rather than the last commit before the deadline, you would run:
git branch -f 123456-mark 123456-main
Passing --create-report will create a csv file with a name of the form
marking-repo-report.csv containing all of the columns from the roster file,
plus the following:
commit_time: The timestamp of the last commit on the student's main (or
master) branch.
late: True if commit_time is after the deadline. This field does take
into account extra time.
cloned: True if the student accepted the assignment on GitHub.
submitted: True if the student pushed at least one commit.
This report is indended to assist in identifying students who may have had technical or other issues.
Passing --push will push all of the local, -main, -master, -feedback,
and -mark branches to the marking repository on GitHub.
Passing --pull-requests will create a pull request for each student in the
marking repository on GitHub. This pull request will compare the -mark branch
with -feedback. Creating pull requests is intentionally throttled to once per
three seconds in order to avoid triggering GitHub rate limits when working with
large classes.
When running a big course, one can easily be overwhelmed by notifications from all of the classroom repositories. It is possible to unwatch everything manually, but that's too much clicking. This can be overcome with:
classroom-tool --unwatch
This unsubscribes the logged-in user from all repositories in the configured classroom that match the configured basename.
Passing --save-feedback will save a PDF of the "Files changed" tab for each
student in the marking repository. Files will be stored in a feedback/
subdirectory with the name of each file being <PR title>.pdf.
Taking these screenshots requires the use of a browser to render the pages and
so selenium is used to automate the process of:
navigating to the correct web page, waiting for it to load and printing to a PDF.
At present only Google Chrome/Chromium (via
chromedriver) is supported.
Each time --save-feedback is passed a fresh browser instance is created
without existing cookies. In order for the browser to be able to access the
pull requests it needs to first be logged in to GitHub. To make this work the
browser will initially load the GitHub login page where the classroom-tool
user can log in to their GitHub account. Once this is done, hitting Enter in the
terminal will trigger classroom-tool to begin saving the PDFs.
Note that the "Files changed" tab on GitHub automatically collapses the diffs from files if the diffs are too large. This means that any comments on the code are not visible. To avoid this assignments should therefore encourage students to not put all of their code into a single large file.
Remapping the repository names to university identity numbers is not enough to anonymise the submissions, because every commit has author information.
Fortunately, git-filter-repo can fix this. The following code will anonymise all commits in the repository:
git-filter-repo --email-callback 'return b"[email protected]"' --force --name-callback 'return b"Anonymous"'
Note that this action will rewrite all the commits in the repository. This means that none of the branches in the marking repository will share any history with the student repositories from which they were copied. This means that you probably only want to take this step once you are sure that all the other steps have been successfully completed. If you have already pushed the branches to GitHub then you will need to force push the anonymised branches:
classroom-tool --push --force