Skip to content

Latest commit

 

History

History
249 lines (186 loc) · 18.6 KB

DeveloperGuide.md

File metadata and controls

249 lines (186 loc) · 18.6 KB
Raw

RepoSense - Developer Guide

Thank you for your interest in contributing to RepoSense!

Setting up

Prerequisites

  1. JDK 1.8.0_60 or later.
  2. git 2.14 or later on the command line.

Type git --version on your OS terminal and ensure that you have the correct version of git.

Setting up the project in your computer using IntelliJ

  1. Fork this repo, and clone the fork to your computer.
  2. Open IntelliJ (if you are not in the welcome screen, click File > Close Project to close the existing project dialog first).
  3. Set up the correct JDK version for Gradle.
    1. Click Configure > Project Defaults > Project Structure.
    2. Click New…​ and find the directory of the JDK.
  4. Click Import Project.
  5. Locate the build.gradle file and select it. Click OK.
  6. Ensure that the selected version of Gradle JVM matches our prerequisite.
  7. Click OK to accept the all the other default settings.

Verifying the setup

  1. Ensure that Gradle builds without error by running the command gradlew clean build, and ensure that it finishs with a BUILD SUCCESSFUL message.
  2. Run the tests to ensure that they all pass by running the command gradlew test systemtest, and ensure that it finishs with a BUILD SUCCESSFUL message.

Ensure that you are on the project root directory when using the gradlew commands.

Configuring the Java coding style

This project follows oss-generic coding standards. IntelliJ’s default style is mostly compliant with our Java coding convention but it uses a different import order from ours. To rectify,

  1. Go to File > Settings…​ (Windows/Linux), or IntelliJ IDEA > Preferences…​ (macOS).
  2. Select Editor > Code Style > Java.
  3. Click on the Imports tab to set the order
    • For Class count to use import with '*' and Names count to use static import with '*': Set to 999 to prevent IntelliJ from contracting the import statements
    • For Import Layout, follow this image below: import-order

Optionally, you can follow the Using Checkstyle document to configure Intellij to check style-compliance as you write code.

Configuring the JavaScript coding style

Our project follows the Airbnb Javascript Style Guide, the eslint configuration file is available at the root of the project. Please run a npm run lint -- --fix frontend/src/**/*js from the project root directory and fix all of the eslint errors before committing your code for final review.

Eslint and its accompaning modules can be installed through NPM, so do ensure that you got it installed if you are working on the dashboard.

Before writing code

  1. Do check out our process guide before submitting any PR with your changes.

Building and running RepoSense from code

  1. Execute the following command on the OS terminal inside the project directory.
    Usage: gradlew run -Dargs="([-config CONFIG_FOLDER] | [-repos REPO_PATH_OR_URL...]) [-view [REPORT_FOLDER]] [-output OUTPUT_DIRECTORY] [-since DD/MM/YYYY] [-until DD/MM/YYYY] [-formats FORMAT...] [-isac | --ignore-standalone-config]"

Sample usage to generate the report with no specify arguments: (find and use config files in current working directory)

gradlew run

Sample usage to generate the report with config files and automatically open the report:

gradlew run -Dargs="-config ./configs/ -output output_path/ -since 21/10/2017 -until 21/11/2017 -formats java adoc js -view"

Sample usage to generate the report with repository locations and automatically open the report:

gradlew run -Dargs="-repos https://github.com/reposense/RepoSense.git https://github.com/se-edu/collate.git -output output_path/ -since 21/10/2017 -until 21/11/2017 -formats java adoc js -view"

Sample usage to generate the report with repository locations but ignore the standalone config file:

gradlew run -Dargs="-repos https://github.com/reposense/RepoSense.git https://github.com/se-edu/collate.git --ignore-standalone-config"

Sample usage to view the report:

gradlew run -Dargs="-view output_path/reposense-report"

-Dargs="..." uses the same argument format as mentioned above.

Architecture

architecture Figure 1. Overall architecture of RepoSense

Parser(ConfigParser)

Parser contains two classes:

  • ArgsParser: Parses the user-supplied command line arguments into a CliArguments object.
  • CsvParser: Parses the the user-supplied CSV config file into a list of RepoConfiguration for each repository to analyze.

Git

Git package contains the wrapper classes for respective git commands.

  • GitBlame: Wrapper class for git blame functionality. Traces the revision and author last modified each line of a file.
  • GitBranch: Wrapper class for git branch functionality. Gets the name of the working branch of the target repo.
  • GitCheckout: Wrapper class for git checkout functionality. Checks out the repository by branch name or commit hash.
  • GitClone: Wrapper class for git clone functionality. Clones the repository from GitHub into a temporary folder in order to run the analysis.
  • GitDiff: Wrapper class for git diff functionality. Obtains the changes between commits.
  • GitLog: Wrapper class for git log functionality. Obtains the commit logs and the authors' info.
  • GitRevList: Wrapper class for git rev-list functionality. Retrieves the commit objects in reverse chronological order.
  • GitShortlog: Wrapper class for git shortlog functionality. Obtains the list of authors who have contributed to the target repo.

CommitsReporter

CommitsReporter is responsible for analyzing the commit history and generating a CommitContributionSummary for each repository. CommitContributionSummary contains information such as each author's daily and weekly contribution and the variance of their contribution. CommitsReporter,

  1. uses CommitInfoExtractor to run the git log command, which generates the statistics of each commit made within date range.
  2. generates a CommitInfo for each commit, which contains the infoLine and statLine.
  3. uses CommitInfoAnalyzer to extract the relevant data from CommitInfo into a CommitResult, such as the number of line insertions and deletions in the commit and the author of the commit.
  4. uses CommitResultAggregator to aggregate all CommitResult into a CommitContributionSummary.

AuthorshipReporter

AuthorshipReporter is responsible for analyzing the white listed files, traces the original author for each line of text/code, and generating an AuthorshipSummary for each repository. AuthorshipSummary contains the analysis results of the white listed files and the amount of line contributions each author made. AuthorshipReporter,

  1. uses FileInfoExtractor to traverse the repository to find all relevant files.
  2. generates a FileInfo for each relevant file, which contains the path to the file and a list of LineInfo representing each line of the file.
  3. uses FileInfoAnalyzer to analyze each file, using git blame or annotations, and finds the Author for each LineInfo.
  4. generates a FileResult for each file, which consolidates the authorship results into a Map of each author's line contribution to the file.
  5. uses FileResultAggregator to aggregate all FileResult into an AuthorshipSummary.

ReportGenerator(Main)

ReportGenerator,

  1. uses GitDownloader API to download the repository from GitHub.
  2. copies the template files into the designated output directory.
  3. uses CommitReporter and AuthorshipReporter to produce the commit and authorship summary respectively.
  4. generates the JSON files needed to generate the HTML dashboard.

System

System contains the classes that interact with the Operating System and external processes.

  • CommandRunner creates processes that executes commands on the terminal. It consists of many git commands.
  • LogsManager uses the java.util.logging package for logging. The LogsManager class is used to manage the logging levels and logging destinations. Log messages are output through: Console and to a .log file.
  • DashboardServer starts a server to display the dashboard on the browser. It depends on the net.freeutils.httpserver package.

Model

Model holds the data structures that are commonly used by the different aspects of RepoSense.

  • Author stores the GitHub ID of an author. Any contributions or commits made by the author, using his/her GitHub ID or aliases, will be attributed to the same Author object. It is used by AuthorshipReporter and CommitsReporter to attribute the commit and file contributions to the respective authors.
  • CliArguments stores the parsed command line arguments supplied by the user. It contains the configuration settings such as the CSV config file to read from, the directory to output the report to, and date range of commits to analyze. These configuration settings are passed into RepoConfiguration.
  • RepoConfiguration stores the configuration information from the CSV config file for a single repository, which are the repository's orgarization, name, branch, list of authors to analyse, date range to analyze commits and files from CliArguments. These configuration information are used by:
    • GitDownloader to determine which repository to download from and which branch to check out to.
    • AuthorshipReporter and CommitsReporter to determine the range of commits and files to analyze.
    • ReportGenerator to determine the directory to output the report.

HTML Dashboard

The source files for the dashboard is located in frontend/src and is built by spuild before being packaged into the JAR file to be extracted as part of the report.

The main HTML file is generated from frontend/src/index.jade.

Vue (pronounced /vjuː/, like view) is a progressive framework for building user interfaces. It is heavily ultilized in the dashboard to dynamically update the information in the various views. (Style guide available here).

dashboard screenshot

Dashboard Architecture

dashboard architecture

The main Vue object (window.app) is responsible for the loading of the dashboard (through summary.json). Its repos attribute is tied to the global window.REPOS, and is passed into the various other modules when the information is needed.

window.app is broken down into two main parts

  • the summary view
  • and the tabbed interface

Summary view act as the main dashboard which shows the various calculations.
Tabbed interface is responsible for loading various modules such as authorship to display additional information.

Javascript Files

  • main.js - main controller that pushes content into different modules
  • api.js- loading and parsing of the dashboard content
  • v_summary.js - module that supports the ramp chart view
  • v_authorship.js - module that supports the authorship view

JSON Report Files

  • summary.json - a list of all the repositories and their respective details
  • projName/commits.json - contains information of the users' commits information (e.g. line deletion, insertion, etc), grouped by date
  • projName/authorship.json - contains information from git blame, detailing the author of each line for all the processed files

Main (main.js)

This contains the logic for main VueJS object, window.app, which is responsible for passing the necessary data into the relevant modules to be loaded.

v_summary and v_authorship are components which will be embedded into dashboard and will render the corresponding content based on the data passed into it from the main window.app.

Loading of dashboard information

The main Vue object depends on the summary.json data to determine the right commits.json files to load into memory. This is handled by api.js which loads the relevant file information from the network files if it is available, otherwise a report archive, archive.zip, have to be used.

Once the relevant commit.json files are loaded, all the repo information will be passed into v_summary to be loaded in the summary view as the relevant ramp charts.

Activating additional view modules

Most activity or actions should happen within the module itself, but in the case where there is a need to spawn or alter the view of another module, an event is emitted from the first module to the main Vue object (window.app), which then handles the data received and passes it along to the relevant modules.

Hash link

Other than the global main Vue object, another global variable we have is the window.hashParams. This object is reponsible for generating the relevant permalink for a specific view of the summary module for the dashboard.

Data loader (api.js)

This is the module that is in charged of loading and parsing the data files generated as part of the report.

Loading from ZIP file

Due to security design, most modern browsers (e.g. Chrome) do not allow web pages to obtain local files using the directory alone. As such, a ZIP archive of the report information will be produced alongside the report generation.

This archive will be used in place of the network files to load information into the dashboard, in the case when the network files are unavailable.

The API module will be handling all request for all the JSON data files. If the network file is not available, the files will be obtained from the zip archive provided.

Retrieving and parsing information

After the JSON files are loaded from their respective sources, the data will be parsed as objects and included inside the global storage object, window.REPOS, in the right format.

For the basic skeleton of window.REPOS, refer to the generated summary.json file in the report for more details.

Summary View (v_summary.js)

The v_summary module is in charge of loading the ramp charts from the corresponding commits.json.

summary architecture

Initializing the data for the ramp charts

The summary module is activated after the information is loaded from the main Vue.JS object. At creation, the repo attribute is populated with the window.REPOS object, which contains information loaded from summary.json.

Filtering users and repositories

The commits information is retrieved from the corresponding project folders for each repository. These information will be filtered and sorted before passed into the template to be displayed as ramp charts.

Padding for dates

For ramps between the date ranges, the slices will be selected and it will be pre and post padded with empty slices to align the ramp slice between the sinceDate and untilDate. The ramps will then be rendered with the slices in the right position.

Authorship View (v_authorship.js)

The authorship module retrieves the relevant information from the corresponding authorship.json file if it is not yet loaded. If it has been loaded, the data will be written into window.REPOS and be read from there instead.

authorship architecture

Showing relevant information by authors

The files will be filtered, picking only files the selected author has written in. The lines are then split into chunks of "touched" and "untouched" code to be displayed in the tab view which will be popped up on the right side of the screen.