Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Documentation Deployment #256

Merged
merged 32 commits into from
Aug 7, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
cb3c01b
Remove old file
nck-mlcnv Jul 24, 2024
a115671
Minor doc change
nck-mlcnv Jul 24, 2024
645738a
Update html documentation page
nck-mlcnv Jul 24, 2024
c3cc84c
Merge branch 'develop' into feature/doc-deployment
nck-mlcnv Jul 26, 2024
0975c59
Remove unused files
nck-mlcnv Jul 26, 2024
a46ff55
Move file
nck-mlcnv Jul 26, 2024
fc04d76
Change image link
nck-mlcnv Jul 26, 2024
4db16dc
Update mkdocs.yml
nck-mlcnv Jul 26, 2024
f06cfe2
Update docs deployment
nck-mlcnv Jul 26, 2024
334c228
Add depyloment test
nck-mlcnv Jul 29, 2024
c795323
Change trigger
nck-mlcnv Jul 29, 2024
d603e0d
Change python action
nck-mlcnv Jul 29, 2024
c105f0e
Change python action 2
nck-mlcnv Jul 29, 2024
0930eaa
Add ontology deployment
nck-mlcnv Jul 29, 2024
44c1177
Fix ontology deployment
nck-mlcnv Jul 29, 2024
856cff1
Fix test
nck-mlcnv Jul 29, 2024
bec03ca
Fix test 2
nck-mlcnv Jul 29, 2024
2486980
Fix test 3
nck-mlcnv Jul 29, 2024
610d98c
Fix test 4
nck-mlcnv Jul 29, 2024
7c2ed54
Fix test 5
nck-mlcnv Jul 29, 2024
02a76a3
Fix test 6
nck-mlcnv Jul 29, 2024
8760187
Fix test 7
nck-mlcnv Jul 29, 2024
419d434
Fix test 8
nck-mlcnv Jul 29, 2024
25d8f28
Fix test 9
nck-mlcnv Jul 29, 2024
422e7d6
Fix test 10
nck-mlcnv Jul 29, 2024
32969eb
Remove test workflow
nck-mlcnv Jul 29, 2024
908ce04
Fix python setup
nck-mlcnv Jul 30, 2024
73c3e84
Test release files
nck-mlcnv Jul 30, 2024
af2c882
Fix workflow
nck-mlcnv Jul 30, 2024
68e1535
Fix workflow 2
nck-mlcnv Jul 30, 2024
612f251
Fix workflow 3
nck-mlcnv Jul 30, 2024
bae0e3e
Remove test workflow
nck-mlcnv Jul 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .bettercodehub.yml

This file was deleted.

9 changes: 0 additions & 9 deletions .github/pages/javadoc-latest.html

This file was deleted.

9 changes: 0 additions & 9 deletions .github/pages/latest.html

This file was deleted.

40 changes: 28 additions & 12 deletions .github/workflows/deploy.yml
bigerl marked this conversation as resolved.
Show resolved Hide resolved
nck-mlcnv marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -86,38 +86,54 @@ jobs:
cache: 'pip'
- run: pip install mkdocs-material
- run: pip install mkdocs-macros-plugin
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml
- run: sed -i "s/\$RELEASE_VERSION/${{ env.RELEASE_VERSION }}/g" mkdocs.yml

- run: mkdocs build -d site/${{ env.RELEASE_VERSION }}
- run: mvn javadoc:javadoc
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/latest.html
- run: sed -i "s/\$VERSION/${{ env.RELEASE_VERSION }}/g" .github/pages/javadoc-latest.html

- name: Deploy Site
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./site/${{ env.RELEASE_VERSION }}
destination_dir: ./docs/${{ env.RELEASE_VERSION }}
- name: Deploy Site
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./site/${{ env.RELEASE_VERSION }}
destination_dir: ./docs/latest

- name: Deploy Javadoc
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./javadoc/${{ env.RELEASE_VERSION }}
publish_dir: ./javadoc/${{ env.RELEASE_VERSION }}/apidocs
destination_dir: ./javadoc/${{ env.RELEASE_VERSION }}
- name: Deploy latest.html
- name: Deploy Javadoc
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: .github/pages/
keep_files: true
destination_dir: ./docs/
- name: Deploy latest.html
publish_dir: ./javadoc/${{ env.RELEASE_VERSION }}/apidocs
destination_dir: ./javadoc/latest

- name: Find Ontology Version
run: echo "ONTOLOGY_VERSION=$(grep 'versionIRI' schema/iguana.owx | grep -Po '[0-9]+.[0-9]+.[0-9]+')" >> $GITHUB_OUTPUT
id: find_ontology_version

- name: Fetch Ontologies
run: git fetch && git checkout origin/gh-pages ontology/
- run: mkdir -p ontology/${{ steps.find_ontology_version.outputs.ONTOLOGY_VERSION }}
- run: cp schema/iguana.owx ontology/${{ steps.find_ontology_version.outputs.ONTOLOGY_VERSION }}/iguana.owx
- run: cp schema/iguana.owx ontology/iguana.owx

- name: Deploy Ontology
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: .github/pages/
keep_files: true
destination_dir: ./docs/
publish_dir: ./ontology/
destination_dir: ./ontology/


deploy_gh_release:
runs-on: ubuntu-latest
Expand Down
Binary file removed customs/images/Iguana_new_logo6.png
Binary file not shown.
Binary file removed customs/images/iguana-result-schema.png
Binary file not shown.
212 changes: 106 additions & 106 deletions docs/README.md
Original file line number Diff line number Diff line change
@@ -1,107 +1,107 @@
<p align="center">
<img src="https://github.com/dice-group/IGUANA/raw/develop/images/IGUANA_logo.png" alt="IGUANA Logo" width="200">
</p>
# IGUANA
Iguana is a benchmarking framework for testing the read performances of HTTP endpoints.
It is mostly designed for benchmarking triplestores by using the SPARQL protocol.
Iguana stresstests endpoints by simulating users which send a set of queries independently of each other.
Benchmarks are configured using a YAML-file, this allows them to be easily repeated and adjustable.
Results are stored in RDF-files and can also be exported as CSV-files.
## Features
- Benchmarking of (SPARQL) HTTP endpoints
- Reusable configuration
- Calculation of various metrics for better comparisons
- Processing of HTTP responses (e.g., results counting)
## Setup
### Prerequisites
If you're using the native version of IGUANA, you need to have at least a `x86-64-v3` (Intel Haswell and AMD Excavator or newer) system that is running Linux.
If you're using the Java version of IGUANA, you need to have `Java 17` or higher installed.
On Ubuntu it can be installed by executing the following command:
```bash
sudo apt install openjdk-17-jre
```
### Download
The latest release can be downloaded at https://github.com/dice-group/IGUANA/releases/latest.
The zip file contains three files:
* `iguana`
* `iguana.jar`
* `example-suite.yml`
* `start-iguana.sh`
The `iguana` file is a native executable for IGUANA that has been compiled with GraalVM.
The `iguana.jar` file is the standard Java executable for IGUANA.
The `start-iguana.sh` script is a helper script to start IGUANA with the `iguana.jar` file.
### Configuration
The `example-suite.yml` file contains an extensive configuration for a benchmark suite.
It can be used as a starting point for your own benchmark suite.
For a detailed explanation of the configuration, see the [configuration](./configuration/overview.md) documentation.
## Usage
### Native Version
Start Iguana with a benchmark suite (e.g., the `example-suite.yml`) by executing the binary:
```bash
./iguana example-suite.yml
```
### Java Version
Start Iguana with a benchmark suite (e.g., the `example-suite.yml`) either by using the start script:
```bash
./start-iguana.sh example-suite.yml
```
or by directly executing the jar-file:
```bash
java -jar iguana.jar example-suite.yml
```
If you're using the script, you can use JVM arguments by setting the environment variable `IGUANA_JVM`.
For example, to let Iguana use 4GB of RAM you can set `IGUANA_JVM` as follows:
```bash
export IGUANA_JVM=-Xmx4g
```
# How to Cite
```bibtex
@InProceedings{10.1007/978-3-319-68204-4_5,
author="Conrads, Lixi
and Lehmann, Jens
and Saleem, Muhammad
and Morsey, Mohamed
and Ngonga Ngomo, Axel-Cyrille",
editor="d'Amato, Claudia
and Fernandez, Miriam
and Tamma, Valentina
and Lecue, Freddy
and Cudr{\'e}-Mauroux, Philippe
and Sequeda, Juan
and Lange, Christoph
and Heflin, Jeff",
title="Iguana: A Generic Framework for Benchmarking the Read-Write Performance of Triple Stores",
booktitle="The Semantic Web -- ISWC 2017",
year="2017",
publisher="Springer International Publishing",
address="Cham",
pages="48--65",
abstract="The performance of triples stores is crucial for applications driven by RDF. Several benchmarks have been proposed that assess the performance of triple stores. However, no integrated benchmark-independent execution framework for these benchmarks has yet been provided. We propose a novel SPARQL benchmark execution framework called Iguana. Our framework complements benchmarks by providing an execution environment which can measure the performance of triple stores during data loading, data updates as well as under different loads and parallel requests. Moreover, it allows a uniform comparison of results on different benchmarks. We execute the FEASIBLE and DBPSB benchmarks using the Iguana framework and measure the performance of popular triple stores under updates and parallel user requests. We compare our results (See https://doi.org/10.6084/m9.figshare.c.3767501.v1) with state-of-the-art benchmarking results and show that our benchmark execution framework can unveil new insights pertaining to the performance of triple stores.",
isbn="978-3-319-68204-4"
}
<p align="center">
<img src="https://github.com/dice-group/IGUANA/raw/main/images/IGUANA_logo.png" alt="IGUANA Logo" width="200">
</p>

# IGUANA
Iguana is a benchmarking framework for testing the read performances of HTTP endpoints.
It is mostly designed for benchmarking triplestores by using the SPARQL protocol.
Iguana stresstests endpoints by simulating users which send a set of queries independently of each other.

Benchmarks are configured using a YAML-file, this allows them to be easily repeated and adjustable.
Results are stored in RDF-files and can also be exported as CSV-files.

## Features
- Benchmarking of (SPARQL) HTTP endpoints
- Reusable configuration
- Calculation of various metrics for better comparisons
- Processing of HTTP responses (e.g., results counting)

## Setup

### Prerequisites

If you're using the native version of IGUANA, you need to have at least a `x86-64-v3` (Intel Haswell and AMD Excavator or newer) system that is running Linux.

If you're using the Java version of IGUANA, you need to have `Java 17` or higher installed.
On Ubuntu it can be installed by executing the following command:

```bash
sudo apt install openjdk-17-jre
```

### Download
The latest release can be downloaded at https://github.com/dice-group/IGUANA/releases/latest.
The zip file contains three files:

* `iguana`
* `iguana.jar`
* `example-suite.yml`
* `start-iguana.sh`

The `iguana` file is a native executable for IGUANA that has been compiled with GraalVM.
The `iguana.jar` file is the standard Java executable for IGUANA.
The `start-iguana.sh` script is a helper script to start IGUANA with the `iguana.jar` file.

### Configuration
The `example-suite.yml` file contains an extensive configuration for a benchmark suite.
It can be used as a starting point for your own benchmark suite.
For a detailed explanation of the configuration, see the [configuration](./configuration/overview.md) documentation.

## Usage

### Native Version

Start Iguana with a benchmark suite (e.g., the `example-suite.yml`) by executing the binary:

```bash
./iguana example-suite.yml
```

### Java Version

Start Iguana with a benchmark suite (e.g., the `example-suite.yml`) either by using the start script:

```bash
./start-iguana.sh example-suite.yml
```

or by directly executing the jar-file:

```bash
java -jar iguana.jar example-suite.yml
```

If you're using the script, you can use JVM arguments by setting the environment variable `IGUANA_JVM`.
For example, to let Iguana use 4GB of RAM you can set `IGUANA_JVM` as follows:

```bash
export IGUANA_JVM=-Xmx4g
```

# How to Cite

```bibtex
@InProceedings{10.1007/978-3-319-68204-4_5,
author="Conrads, Lixi
and Lehmann, Jens
and Saleem, Muhammad
and Morsey, Mohamed
and Ngonga Ngomo, Axel-Cyrille",
editor="d'Amato, Claudia
and Fernandez, Miriam
and Tamma, Valentina
and Lecue, Freddy
and Cudr{\'e}-Mauroux, Philippe
and Sequeda, Juan
and Lange, Christoph
and Heflin, Jeff",
title="Iguana: A Generic Framework for Benchmarking the Read-Write Performance of Triple Stores",
booktitle="The Semantic Web -- ISWC 2017",
year="2017",
publisher="Springer International Publishing",
address="Cham",
pages="48--65",
abstract="The performance of triples stores is crucial for applications driven by RDF. Several benchmarks have been proposed that assess the performance of triple stores. However, no integrated benchmark-independent execution framework for these benchmarks has yet been provided. We propose a novel SPARQL benchmark execution framework called Iguana. Our framework complements benchmarks by providing an execution environment which can measure the performance of triple stores during data loading, data updates as well as under different loads and parallel requests. Moreover, it allows a uniform comparison of results on different benchmarks. We execute the FEASIBLE and DBPSB benchmarks using the Iguana framework and measure the performance of popular triple stores under updates and parallel user requests. We compare our results (See https://doi.org/10.6084/m9.figshare.c.3767501.v1) with state-of-the-art benchmarking results and show that our benchmark execution framework can unveil new insights pertaining to the performance of triple stores.",
isbn="978-3-319-68204-4"
}
```
1 change: 1 addition & 0 deletions docs/configuration/workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,5 +132,6 @@ If the property is set to `false`,
the worker will not parse the response bodies and will not calculate hash values for the response bodies.

Setting the property to `false` can improve the performance of the worker.
This means that the worker is able to measure the performance more accurately.
If the property is set to `true`, the worker will temporarily store the whole response bodies in memory for processing.
If the property is set to `false`, the worker will discard any received bytes from the response.
Binary file removed images/iguana3-logo.png
Binary file not shown.
File renamed without changes
Loading
Loading