benchbase: perf-data-loader: array data type support

Prerequisites

• JDK17 or higher
• maven 3.6 or higher
• Git recent version

---

Quickstart (release build)

wget https://github.com/yugabyte/benchbase/releases/download/1.16.4/benchbase-yugabyte.tgz

This will download benchbase-yugabyte.tgz, which can be extracted,

tar xvzf benchbase-yugabyte.tgz
cd benchbase-yugabyte

Inside this folder, you can run Featurebench with your YAML. For example, to execute a microbenchmark,

java -jar benchbase.jar -b featurebench -c config/yugabyte/sample_featurebench_config.yaml --create=true --load=true --execute=true

---

Quickstart (source code)

To clone and build BenchBase using the yugabyte profile,

git clone --depth 1 https://github.com/yugabyte/benchbase.git
cd benchbase
./mvnw clean package -P yugabyte -DskipTests
# you can also use ./build.sh 

This produces artifacts in the target folder, which can be extracted,

cd target
tar xvzf benchbase-yugabyte.tgz
cd benchbase-yugbayte

Inside this folder, you can run Featurebench with your YAML. For example, to execute a microbenchmark,

java -jar benchbase.jar -b featurebench -c config/yugabyte/sample_featurebench_config.yaml --create=true --load=true --execute=true

Perf Dataloader

Used to infer the table schema from the database and generate a benchbase interpreted input yaml file which can be used to load sample data into the table.

This tool is integrated inside benchbase so that the users don't have to install additional tools for using it. All existing functionalities from yugabyte/benchbase should work as it is.

How to use:

#$./perf-data-loader --help
Usage: ./perf-data-loader --config <config_file> --table-name <table_name> --rows <rows> [--gen-config-only] [--load-only]
Short forms: -c <config_file> -t <table_name> -r <rows>
Options:
  -c, --config                Configuration file
  -t, --table-name            Table name
  -r, --rows                  Number of rows
  --gen-config-only           Only generate the loader/config file
  --load-only                 Only load data into the database
  -h, --help                  Display this help message

• to only generate the loader file(skip the actual load). This will generate the yaml file _loader.yaml which can be used in loading the data.

./perf-data-loader --config <config_file> --table-name <table_name> --rows <rows> --gen-config-only

• to only load the data(when your loader file is already generate)

./perf-data-loader --config <config_file> --load-only

• to generate the loader yaml file and load the data in one go

./perf-data-loader --config <config_file> --table-name <table_name> --rows <rows>

• to generate the load order of all the tables in the database. It is useful to know the load order when there are
  foreign key references in the db schema. When used, this command will print out the load order/levels in which you can
  populate the tables. Start from tables at level 0, then level 1, and so on. It also generate load_order.json file
  containing the load order in json format.

./perf-data-loader -c <config_file> --gen-load-order

the input yaml file should have following content

type: YUGABYTE
driver: com.yugabyte.Driver
url: jdbc:yugabytedb://localhost:5433/yugabyte?sslmode=require&reWriteBatchedInserts=true
username: yugabyte
password: password

tablename: {{tableName}}
rows: {{rows}}

Caveat/In-progress items for perf-data-loader

• partitioned tables are not yet supported.
• columns with user defined data types are not yet supported.

Reference utility functions