Benchmarking AnzoGraph Performance

This topic provides information about running benchmark tests that measure the performance of AnzoGraph by running a variety of queries against data at scale.

This topic provides instructions for running the performance benchmark, GHIB, that is released in the AnzoGraph deployment. For best practices and in-depth information about benchmarking with AnzoGraph DB, see the AnzoGraph DB Benchmarking Guide.

The GHIB Benchmark

The Graph ad Hoc business Intelligence Benchmark, GHIB, measures the timing of 22 different queries executed against a synthetic data set. The executed queries were ported from the TPC-H benchmark and operate on a generated data set that is compatible with TPC-H. The TPC Benchmark™H (TPC-H) is a decision support benchmark that consists of a suite of business-oriented ad hoc queries and concurrent data modifications. This benchmark illustrates decision support systems that examine large volumes of data, execute queries with a high degree of complexity, and give answers to critical business questions.

Background

The file system of an AnzoGraph deployment includes a set of scripts (located in the etc/ghib directory) that enable customers to generate a data set called GHIB that represents the TPC-H Decision Support data originally designed for SQL queries. The data set captures activities for wholesale suppliers, where vendors purchase parts from suppliers, and sell those parts to customers. For more information about the data design, see Working with the TPCH Data.

Running the GHIB scripts also generates CSV load files, loads the files into AnzoGraph, and then executes the 22 GHIB queries contained in the etc/ghib directory. The scripts return the number of triples that were created, the time it took to load them, and the execution time for each query. The queries are each run twice, first in a warm-up run which includes compile time, and a second run that computes benchmark results with the pre-compiled queries. Timing results from the second run are used for benchmarking comparisons. Following their execution, the GHIB scripts also create an xray that can be submitted to Cambridge Semantics Support if any problem is suspected.

Scale Factors

Users running the GHIB benchmark tests can generate test data specifying different scale factors to increase the number of triples to test:

With a scale factor of 1, the GHIB data set approximately ~105 million triples.
A scale factor of 10 creates a data set of roughly 1 billion triples.
A scale factor 30 creates a data set of roughly 3 billion triples, and so on.

The whole GHIB benchmark test can take some time to run, especially if large scale factors are chosen.

Prerequisites

AnzoGraph must be running when the test is started.

Appropriate disk space and memory must also be available, based on the size of the data set you choose for benchmark testing. A scale factor of 1 produces about 1.1 GB of data in the AnzoGraph file system and generally requires around 16 GB of available memory to store the data and run the queries.

Running the test

To run the GHIB benchmark, first make sure that AnzoGraph is running and then run the following command:

./etc/ghib/ghib_all.sh <directory to store data> <scale_factor>

For example, the following command runs the benchmark scripts with a scale factor of 1, and instructs AnzoGraph to create a "ghib" directory in /tmp: