Introduction to Data Layers

Data Layers enhance Graphmarts dynamically by loading additional Datasets, masking certain data, inferring new data automatically, or running SPARQL queries to create, clean, conform, transform, or validate data. This topic introduces the fundamental concepts to know when working with Data Layers.

What is a Data Layer?

A Data Layer is a container for one or more steps. The steps in a layer perform the data loads or creation and transformation.

  • You can create any number of layers in a Graphmart and control which users and groups have access to the layers and the data within the layers.
  • Users can toggle Data Layers on and off in Hi-Res Analytics dashboards.
  • You can configure Masking on a layer to hide sensitive information.
  • You can configure Execution Conditions on Data Layers so that the layer is processed only when certain conditions are met.
  • You cannot share Data Layers between Graphmarts, but you can clone layers and include a copy in multiple Graphmarts.
  • You control the source data for steps in a layer. Layers can build upon the data generated by steps in previous layers or can be self-contained, applying changes only to the data defined in the layer.

For instructions on creating Data Layers, see Adding Data Layers to Graphmarts.

What is a Step?

The Steps in a layer perform all of the data operations, such as loading a Dataset, creating new data, or transforming the existing data. You can add any number of Steps to a layer and can create the following types of steps:

  • Export Step: Exports the Graphmart data in memory to a file-based linked data set (FLDS).
  • Load Data Step: Loads a Dataset from the Dataset catalog into a Data Layer graph. When you create a Graphmart from the catalog, Anzo automatically creates a Data Layer with a Load Data Step.
  • Pre-compile Query Step: Runs the included query immediately after a Graphmart is loaded so that the query is pre-compiled by AnzoGraph. Pre-compiling a query reduces execution time when a user runs that query for the first time.
  • Query Driven Templated Step: Enables users to create reusable query-driven templates for quickly creating additional query steps. Unlike the Templated Step, where users define each key-value pair, this step runs a query to identify all of the key-value pairs. Then the template query is run for each key-value solution from the first query.
  • Query Step: Provides a SPARQL query template that you can use for writing a query that creates, cleans, conforms, or transforms data in the Data Layer.
  • RDFS+ Inference Step: Uses RDFS and OWL rules to generate new data in a layer based on the vocabularies in the existing data.
  • Templated Step: Enables users to create reusable templates for quickly creating additional query steps. The query in a Templated Step uses parameters to represent key-value pairs. When reusing the step, users modify the values for the keys rather than rewriting the query.
  • Validation Step: Enables users to write a query that validates the data in a Data Layer.
  • View: Enables advanced users to write a SPARQL CONSTRUCT query that defines a view of the data but does not alter the source data or create new data unless you choose to materialize the view.

For information about creating steps, see Adding Steps and Views to Data Layers.

Related Topics