Sizing Guidelines for In-Memory Storage

This topic provides guidance on determining the server and cluster size that is ideal for hosting AnzoGraph, depending on the characteristics of your data and AnzoGraph use case.

• Memory and Cluster Size Guidelines
• Analyzing Data Characteristics in Load Files
• Estimating Memory Requirements Based on Data Characteristics

Memory and Cluster Size Guidelines

Since AnzoGraph is a high-performance, in-memory database, it is important to consider the amount of memory needed to store the data that you plan to load. Estimating the amount of memory your workload requires can help you decide what size server to use and whether to use multiple servers. The sections below describe the key points to consider about memory usage and AnzoGraph.

• Data at rest should use less than 50% of the total memory
• AnzoGraph reserves 20% of the memory for the OS
• Memory usage depends on data characteristics
• Memory usage can be high during loads
• Memory Usage Examples

When your workload size requires using a cluster, do not create clusters with fewer than 4 nodes. When using a single node, data gets redistributed in memory without using the network. If you add 1 or 2 more nodes to create a 2- or 3-node cluster, data then gets distributed over the network. The CPU gain from the additional 1 or 2 nodes does not outweigh the performance degradation from the network. Using at least 4 nodes significantly reduces the network degradation and provides a near-linear performance benefit when compared to a single node.

Data at rest should use less than 50% of the total memory

The data loaded into memory should not consume more than 50% of the total available memory on the instance or across a cluster. Preserve at least 50% of the memory for server processes, query processing, and storing intermediate results.

AnzoGraph reserves 20% of the memory for the OS

To avoid unexpected shutdowns by the Linux operating system, the default AnzoGraph configuration leaves 20% of memory available for the OS; AnzoGraph will not use more than 80% of the total available memory. Account for this memory buffer in sizing calculations.

Memory usage depends on data characteristics

Memory usage varies significantly depending on the makeup of the data, such as the data types and sizes of literal values, and the complexity of the queries that you run. Data is loaded into AnzoGraph as triples, and the storage required for each triple ranges anywhere from 12 bytes per triple to 1 megabyte, for a triple that stores pages of text from an unstructured document.

• Triples with integer objects like the following example require about 16 bytes to store in memory.

  <http://anzograph.com/resource/person1> <http://anzograph.com/resource/age> 50

• Triples made up of URIs like the following example require about 18 bytes to store in memory.

  <http://anzograph.com/resource/person1> <http://anzograph.com/resource/friend> <http://anzograph.com/resource/person100>

• Triples with user-defined data types (UDTs) like the following example also require about 18 bytes to store in memory.

<http://anzograph.com/resource/person1> <http://anzograph.com/resource/height> "5'8""^^height

• Triples with dateTime values like the following example require about 20 bytes to store in memory.

  <http://www.wikidata.org/entity/Q65949130> 
  <http://www.wikidata.org/prop/direct/P585> 
  "1995-01-01T00:00:00Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> .

• Triples with long strings like the following example require about 700 bytes to store in memory.

  <http://dbpedia.org/resource/Keanu_Reeves> <http://dbpedia.org/ontology/abstract> "Keanu Charles Reeves
  (/keɪˈɑːnuː/ kay-AH-noo; born September 2, 1964) is a Canadian actor, producer, director and musician.
  Reeves is best known for his acting career, beginning in 1985 and spanning more than three decades.
  He gained fame for his starring role performances in several blockbuster films including comedies
  from the Bill and Ted franchise (1989–1991), action thrillers Point Break (1991) and Speed (1994),
  and the science fiction-action trilogy The Matrix (1999–2003). He has also appeared in dramatic
  films such as Dangerous Liaisons (1988), My Own Private Idaho (1991), and Little Buddha (1993),
  as well as the romantic horror Bram Stoker's Dracula (1992)."

The following table provides estimates for the number of triples that you can load and query with specific amounts of available RAM. The table also lists the number of triples that could be stored in given amounts of memory, using the triples described in the previous examples.

Memory usage can be high during loads

During the load process, before the data can be moved to its final storage block, memory usage temporarily increases, particularly if the data includes many string values. For example, though you can expect to be able to store and query about one billion triples on a server with 200 GB available RAM, loading all one billion triples in a single load command might require more memory than is available on the server. For optimal performance and to minimize memory usage, break data into several smaller load files and use multiple load statements to load the files in batches. For more information about staging load files, see Load File Requirements

Memory Usage Examples

The AnzoGraph installation includes two sample data sets. Combined, the data sets have about 110 million triples. The data originates from structured data sources and contains many URIs, short strings, and numeric values. After the load, the data consumes about 3 GB of available memory. While the load is running, however, AnzoGraph uses up to 7 GB of memory. Accounting for the 20% of memory reserved for the operating system, loading the 110 million sample triples requires an instance with at least 9 GB of memory available at load time. Considering that a user might run very complex queries or advanced analytics against all of the data, an instance with 16 – 32 GB of available memory is the ideal size when working with the sample data or similar data sets.

The following table provides memory requirement estimates for various additional public or commercial data sets that customers may already use or be familiar with:

Analyzing Data Characteristics in Load Files

AnzoGraph allows you to perform pre-load analysis on load files without actually loading the data into memory. You can use this method to run statistical queries, such as counting the number of triples, getting to know the data, or returning a list of the nodes or subjects and predicates. Performing a "dry run" of a data load, beforehand, enables you to analyze data set characteristics to help with tasks such as memory sizing and overall capacity planning. You can use this method to capture statistics about a large data set on a smaller system than what would actually be required to load the data in memory.

Important Considerations for Analyzing Load Files

• Since AnzoGraph scans the files on-disk, queries run much slower than they do when run against data in memory. Consider performance when deciding how many files to query at once and how complex to make queries.
• Though the pre-load feature does not use memory for storing data, queries that you run against files do consume some memory. The server must have sufficient memory available to use for these intermediate query results.
• Unlike loads into the database, pre-load analysis does not prune duplicate triples. Statistics returned for load file queries may differ somewhat from the statistics returned after the data is loaded.

Analysis Query Syntax

The syntax that you use to query load files depends on the file type. For example, for files in triple or quad format, like Turtle (.ttl), N-Triple (.n3 and .nt), N-Quad (.nq and .quads), and TriG (.trig) files, you can use the following syntax:

SELECT expression
FROM EXTERNAL <dir:/path/dir_or_file_name>
[ FROM EXTERNAL <dir:/path/dir_or_file_name> ]
WHERE { triple_patterns }

<file:/home/user/data/tickit.ttl>

<dir:/data/load-files/tickit.ttl.gz>

For example, the following query analyzes the tickit.ttl.gz directory to count the total number of triples in the files:

SELECT (count (*) as ?triples)
FROM EXTERNAL <dir:/opt/anzograph/etc/tickit.ttl.gz>
WHERE { ?s ?p ?o . }

triples
--------
5368800
1 rows

Estimating Memory Requirements Based on Data Characteristics

Although the memory required to load and perform queries on specific data sets will vary based on the size and type of data contained in a data set, you can still obtain a reasonable estimate or starting point for the amount of memory you will need to load any specific data set. Using the method of pre-load analysis of load files described earlier (see Analyzing Data Characteristics in Load Files), you can query the data set to calculate a rough estimate of the memory required to load the data set in memory.

1. Calculate the number of triples the data set will generate when stored in AnzoGraph.
2. Multiply the number of triples by an average triple size.
3. Add the number of characters stored in all of the character strings contained in the data set.

Using the example of the Tickit data set provided with AnzoGraph, you can perform a query like the following to calculate the number of triples the Tickit data set will contain when loaded into AnzoGraph:

SELECT (COUNT(*) as ?triple_count)
FROM EXTERNAL <dir:/install_path/etc/tickit.ttl.gz>
WHERE {?s ?p ?o}

triple_count
------------
7696012
1 rows

Once you know the total number of triples, multiply the value by the average triple storage size. The Memory usage depends on data characteristics section above shows some example triples and their estimated size. If you are familiar with the data in the files, you may be able to determine the average size based on the examples. Otherwise, Cambridge Semantics recommends using 30 bytes as the average triple size. For example, using the triple count above and an average triple size of 30 bytes:

7696012 x 30 = 230,880,360 bytes

To calculate the additional memory required for in-memory storage of character string data, you can run a query like the following:

SELECT
  (SUM(IF(DATATYPE(?o)=<http://www.w3.org/2001/XMLSchema#string>,(STRLEN(?o)),0)) AS ?char_count)
FROM EXTERNAL <dir:/install_path/etc/tickit.ttl.gz>
WHERE {?s ?p ?o.}
 
char_count 
----------- 
4893660 
1 rows

For ASCII characters, AnzoGraph requires a single byte of memory for each character, so adding the total number of characters to the previous memory calculation for storing triples, the result is the following:

230,880,360 + 4,893,660 = 235,774,020 total bytes

Note that the calculation of 235,774,020 total bytes (0.24 GB) provides an estimate for data set storage "at rest" and takes into account only one data set stored in memory. So, in coming up with a final recommendation for total memory requirements of an AnzoGraph deployment, you need to account for any other data sets you may want to load in memory at the same time. You also need to keep in mind other memory sizing guidelines, for example, that all loaded data should not consume more than 50% of all available RAM.

Related Topics

• Server and Cluster Requirements
• Sizing Guidelines for Disk-Based Storage (Preview)
• Deploying AnzoGraph