IGUANA
Iguana is a benchmarking framework for testing the read performances of HTTP endpoints. It is mostly designed for benchmarking triplestores by using the SPARQL protocol. Iguana stresstests endpoints by simulating users which send a set of queries independently of each other.
Benchmarks are configured using a YAML-file, this allows them to be easily repeated and adjustable. Results are stored in RDF-files and can also be exported as CSV-files.
Features
- Benchmarking of (SPARQL) HTTP endpoints
- Reusable configuration
- Calculation of various metrics for better comparisons
- Processing of HTTP responses (e.g., results counting)
Setup
Prerequisites
If you're using the native version of IGUANA, you need to have at least a x86-64-v3
(Intel Haswell and AMD Excavator or newer) system that is running Linux.
If you're using the Java version of IGUANA, you need to have Java 17
or higher installed.
On Ubuntu it can be installed by executing the following command:
sudo apt install openjdk-17-jre
Download
The latest release can be downloaded at https://github.com/dice-group/IGUANA/releases/latest. The zip file contains three files:
iguana
iguana.jar
example-suite.yml
start-iguana.sh
The iguana
file is a native executable for IGUANA that has been compiled with GraalVM.
The iguana.jar
file is the standard Java executable for IGUANA.
The start-iguana.sh
script is a helper script to start IGUANA with the iguana.jar
file.
Configuration
The example-suite.yml
file contains an extensive configuration for a benchmark suite.
It can be used as a starting point for your own benchmark suite.
For a detailed explanation of the configuration, see the configuration documentation.
Usage
Native Version
Start Iguana with a benchmark suite (e.g., the example-suite.yml
) by executing the binary:
./iguana example-suite.yml
Java Version
Start Iguana with a benchmark suite (e.g., the example-suite.yml
) either by using the start script:
./start-iguana.sh example-suite.yml
or by directly executing the jar-file:
java -jar iguana.jar example-suite.yml
If you're using the script, you can use JVM arguments by setting the environment variable IGUANA_JVM
.
For example, to let Iguana use 4GB of RAM you can set IGUANA_JVM
as follows:
export IGUANA_JVM=-Xmx4g
How to Cite
@InProceedings{10.1007/978-3-319-68204-4_5,
author="Conrads, Lixi
and Lehmann, Jens
and Saleem, Muhammad
and Morsey, Mohamed
and Ngonga Ngomo, Axel-Cyrille",
editor="d'Amato, Claudia
and Fernandez, Miriam
and Tamma, Valentina
and Lecue, Freddy
and Cudr{\'e}-Mauroux, Philippe
and Sequeda, Juan
and Lange, Christoph
and Heflin, Jeff",
title="Iguana: A Generic Framework for Benchmarking the Read-Write Performance of Triple Stores",
booktitle="The Semantic Web -- ISWC 2017",
year="2017",
publisher="Springer International Publishing",
address="Cham",
pages="48--65",
abstract="The performance of triples stores is crucial for applications driven by RDF. Several benchmarks have been proposed that assess the performance of triple stores. However, no integrated benchmark-independent execution framework for these benchmarks has yet been provided. We propose a novel SPARQL benchmark execution framework called Iguana. Our framework complements benchmarks by providing an execution environment which can measure the performance of triple stores during data loading, data updates as well as under different loads and parallel requests. Moreover, it allows a uniform comparison of results on different benchmarks. We execute the FEASIBLE and DBPSB benchmarks using the Iguana framework and measure the performance of popular triple stores under updates and parallel user requests. We compare our results (See https://doi.org/10.6084/m9.figshare.c.3767501.v1) with state-of-the-art benchmarking results and show that our benchmark execution framework can unveil new insights pertaining to the performance of triple stores.",
isbn="978-3-319-68204-4"
}