Reading Remote Source Metadata

If you want to retrieve instance data from a source but are unsure about the data model, schema, or the exact names of columns and their data types, you can use the Graph Data Interface (GDI) service to explore the source's metadata. The service can be used to return a list of the catalogs (schemas), models, columns, data types, and other data source specific information.

Graph Data Interface service calls are processed by AnzoGraph using a Java plugin. Before running GDI service queries, make sure that the AnzoGraph cluster is configured to use the plugin. For more information, see Graph Data Interface Setup.

This topic describes the metadata query syntax and provides several example queries.

Metadata Query Syntax
Metadata Query Examples

Metadata Query Syntax

The following query syntax shows the structure of a metadata query. The clauses, patterns, and placeholders in blue are described below.

# PREFIX Clause 
PREFIX s:      <http://cambridgesemantics.com/ontologies/DataToolkit#>
PREFIX rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:   <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd:    <http://www.w3.org/2001/XMLSchema#>
PREFIX owl:    <http://www.w3.org/2002/07/owl#>
PREFIX anzo:   <http://openanzo.org/ontologies/2008/07/Anzo#>
PREFIX zowl:   <http://openanzo.org/ontologies/2009/05/AnzoOwl#>
PREFIX dc:     <http://purl.org/dc/elements/1.1/> 

# Result Clause
SELECT *
WHERE
{
	SERVICE <http://cambridgesemantics.com/services/DataToolkit>
    {	
	   [] s:select ?metadata .

	   ?data a s:source_type ;
		  s:connection_parameters .

	   ?metadata a s:MetadataSource ;
		  s:from ?data ;
		
	   ?metadata_selector [
		  ?metadata_type (datatype) ;
		  ... ;
	   ] .
   }
}

PREFIX Clause

The PREFIX clause declares the prefixes that are standard for all GDI service queries. You can declare additional prefixes to use in the query, but the PREFIX clause must include the following statements:

PREFIX s:      <http://cambridgesemantics.com/ontologies/DataToolkit#>
PREFIX rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:   <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd:    <http://www.w3.org/2001/XMLSchema#>
PREFIX owl:    <http://www.w3.org/2002/07/owl#>
PREFIX anzo:   <http://openanzo.org/ontologies/2008/07/Anzo#>
PREFIX zowl:   <http://openanzo.org/ontologies/2009/05/AnzoOwl#>
PREFIX dc:     <http://purl.org/dc/elements/1.1/>

Result Clause

The result clause defines the results to return. For metadata queries, the result clause is typically SELECT *.

SERVICE <http://cambridgesemantics.com/services/DataToolkit>

Include the required GDI SERVICE call in the WHERE clause. The rest of the WHERE clause defines the patterns to look for in the source.

[] s:select ?metadata

Include this required triple pattern in metadata queries. The select property specifies the source that should be used to return data.

source_type

The ?data a s:source_type triple pattern specifies the type of data source that the query targets. For example, ?data a s:DbSource targets a database data source. The list below describes the commonly used types:

DbSource for any type of database.
FileSource for flat files. The supported file types are CSV and TSV, JSON, XML, Parquet, and SAS (SAS Transport XPT and SAS7BDAT formats) .
HttpSource for HTTP endpoints.

For a complete list of the supported source types, view the GDI ontology. Each data source type is represented by an owl:Class. See Getting Familiar with the Graph Data Interface for more information.

connection_parameters

The source connection parameters are the values that are required for accessing the source, such the database connection URL, path to a file source, username, password, key, token, etc.

For example, the pattern below specifies the connection details for a database source:

?data a s:DbSource ;
   s:url "jdbc:postgresql://10.100.2.9:5555/kl_hosp_db?
user=postgres&password=postgres123"

The example below specifies the connection details for a file-based source, a directory of CSV files:

?data a s:FileSource ;
   s:url "/opt/shared-files/sales-csv"

For a single file, specify the filename in the URL. For example, "/opt/shared-files/sas/edu_inc.sas7bdat".

metadata_selector

The rest of the WHERE clause defines the metadata to retrieve. The metadata_selector specifies the type of metadata to return. The following list describes the valid selectors:

catalogs: This selector narrows the results to schema-related metadata such as the schema names. Even when additional metadata types (described in the row below) are specified as objects, only catalog (schema) information is returned.
fields: This selector is the broadest and most flexible option. Using the fields selector enables users to return any and all of the source metadata information, depending on the specified metadata types (described in the row below).
models: This selector narrows the results to model-related metadata such as the model names. Even when additional metadata types (described in the row below) are specified as objects, only model information is returned.

metadata_type (datatype)

The triple patterns in the array for the metadata selector specify the type of metadata to return as well as the data type for the return value.

The following list shows all of the valid options. You can include any combination of properties. The results that are returned depend on the type of data source and whether the information exists in the source.

?model (xsd:string): Returns model names in string format. For file sources, this property returns file names.
?field (xsd:string): Returns column names.
?catalog (xsd:string): Returns schema names.
?datatype (owl:Thing): Returns the data types of the columns.
?format (xsd:string): Returns the format of the source.
?cardinality (xsd:string): Returns the cardinality of relationships between tables: optional, many, or required.
?count (xsd:int): Returns the number of times the field appears in the source.
?order (xsd:int): Returns the order in which the field was encountered.

The Metadata Query Examples section below provides sample metadata queries that access various data sources.

Metadata Query Examples

This section includes sample metadata queries that run against different types of data sources.

View Database Schemas
Explore a Database Schema
Explore a Directory of SAS Files
Explore an HTTP Endpoint
Explore a Directory of CSV Files

View Database Schemas

The query below sends a metadata query to a MySQL database to return a list of the schemas that are available:

PREFIX s:      <http://cambridgesemantics.com/ontologies/DataToolkit#>
PREFIX rdf:    <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs:   <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd:    <http://www.w3.org/2001/XMLSchema#>
PREFIX owl:    <http://www.w3.org/2002/07/owl#>
PREFIX anzo:   <http://openanzo.org/ontologies/2008/07/Anzo#>
PREFIX zowl:   <http://openanzo.org/ontologies/2009/05/AnzoOwl#>
PREFIX dc:     <http://purl.org/dc/elements/1.1/>

SELECT *
WHERE
{
	SERVICE <http://cambridgesemantics.com/services/DataToolkit>
    {
	   [] s:select ?metadata .

	   ?data a s:DbSource ;
		 s:url "jdbc:mysql://10.100.2.9:5555/?user=root&password=Mysql1@#" .

	   ?metadata a s:MetadataSource ;
		 s:from ?data ;
	   ?catalogs [
		 ?catalog xsd:string ;
		 ?order xsd:int ;
	   ] .
   }
}
ORDER BY ?catalog