Tutorial: Building a Sample Solution from Scratch
This topic helps get you started with Anzo by teaching you the basic steps to follow when creating a complete solution. The instructions below provide a sample CSV file and guide you through the steps required from importing the file to visualizing the data in a Hi-Res Analytics dashboard.
- Prepare the Environment
- Download the Sample Data
- Create a CSV Data Source and Import the Sample Data
- Ingest the Imported Data Set
- Run the ETL Pipeline
- Create and Activate the Movie Data Graphmart
- Explore the Data in Hi-Res Analytics
- Review the New Solution's Artifacts
To give the tutorial continuity and make it straightforward to follow, it helps to complete a few general setup steps before you start building the solution. This section provides background information and describes the environment preparation.
This tutorial guides you through importing data from a file, running an ETL pipeline to create RDF files for AnzoGraph, and loading the data into AnzoGraph. Completing those steps requires a file store that Anzo and AnzoGraph can access, a designated location (Anzo data store) on that storage system to place the RDF files that Spark generates, and a connection to AnzoGraph for loading the data.
Confirm that the following setup steps are complete before proceeding:
- AnzoGraph and AnzoGraph have access to the same file store and its location is configured in Anzo. For instructions, see Connecting to a File Store.
The Anzo server file system is configured and accessible by default. If you store files on a file system that is mounted directly to the Anzo and AnzoGraph servers, you do not need to configure a new file system location.
- An Anzo data store has been created for the RDF files that are generated during the ETL process. For instructions on creating a data store, see Creating an Anzo Data Store.
- Anzo has a connection to AnzoGraph. For instructions on configuring the connection, see Making a Basic Connection to AnzoGraph.
This tutorial uses a small sample data set from Data World. The data is in CSV format and includes IMDB movie data from 2006 – 2016. Click the link below to download IMDB-Movie-Data.zip to your computer. Then extract the ZIP file to make IMDB-Movie-Data.csv available for importing.
- In the Anzo application, expand the Onboard menu and click Structured Data. Anzo displays the Data Sources screen, which lists any existing data sources:
- Click the Add Data Source button and select File Data Source > CSV Data Source. Anzo opens the Create CSV Data Source screen.
- Type a name for the source in the Title field. For example, "Sample Movie Data." Then type an optional description in the Description field. For example:
- Click Save. Anzo saves the source and displays the Tables tab for the new data source.
- On the left side of the screen, click the Add New File button. Anzo displays the Add New File dialog box, and the Upload Files tab is selected.
- On the Upload Files tab, drag and drop the IMDB-Movie-Data.csv file onto the screen or click browse and navigate to the file and double-click it to select it. Anzo attaches the file and the Next button becomes active. Click Next. Anzo lists the file on the left side of the screen with a status of Pending. For example:
- Click the Process Pending Files button to import the IMDB data to Anzo. Anzo imports the data and the status changes to Processed. Additional buttons are also displayed on the right side of the screen. You can click the table row for IMDB-Movie-Data to display the schema details.
The data is now imported as a CSV data source.
Proceed to the next step to ingest the data and automatically derive a data model, generate a mapping, and create an ETL pipeline for transforming the data to the RDF graph model.
- On the Tables screen (from the last step of the procedure above), click the Ingest button. Anzo opens the Ingest dialog box and automatically populates the data source value. If there is only one configured data store, the Anzo Data Store value is also auto-populated. In addition, if the default ETL Engine is configured for the system, the Auto Map Engine Config field will also be populated. For example, in the image below the Anzo Data Store and Auto Map Engine Config fields are not populated because there are two available choices:
- If necessary, click the Anzo Data Store field and select the data store for this pipeline.
- If necessary, click the Auto Map Engine Config field and select one of the ETL engines to use. For this tutorial, select Local Sparkler Engine. That means the pipeline will use the SPARQL-driven Sparkler ETL compiler to process the source data for the Spark engine.
- Leave the Select all tables radio button selected. For example, the image below shows the completed screen:
- Click Save. Anzo creates a pipeline for the data set and displays a "Pipeline was successfully created" confirmation. Click OK to close the dialog box.
As part of creating the pipeline, Anzo also generates a data model and mapping for the sample data set. The last step in this tutorial describes each of the components. Proceed to the next step to run the pipeline and generate the RDF File-Based Linked Data Set (FLDS).
- In the Onboard menu, click Structured Data. Then click the Pipelines tab. Anzo displays the Pipelines screen, which lists the pipeline for the sample movie data set. For example:
- Click the pipeline title to open the pipeline Overview tab. For example:
- Click the Publish All button to run the pipeline, which transforms the data to RDF graph format and generates the FLDS in the specified data store.
Anzo publishes the ETL files and displays a confirmation that job execution is in progress. Click OK to close the dialog box. Anzo completes the job execution and the sample movie data set becomes available in the Dataset catalog. Proceed to the next step to create a graphmart for the new data and load it into AnzoGraph.
- In the Anzo application, expand the Blend menu and click Datasets. Anzo displays the Datasets catalog, which shows an inventory of all of the onboarded data in Anzo. It lists the sample movie data set. For example:
- Hover the pointer over the Sample Movie Data item to display a checkbox in the left column, and then select the checkbox. Anzo adds the data set to the shopping cart and additional icons become available at the top of the screen. For example:
- Click the shopping cart icon () at the top of the screen. Anzo displays the Create Graphmart screen. For example:
Anzo populates the Title field by appending "Graphmart" to the data set name.
- On the Create Graphmart screen, you have the option edit the title and type an optional description. Click Save when you are ready to create the graphmart. Anzo creates the graphmart and displays the Overview screen for the new graphmart. For example:
- To load the graphmart to AnzoGraph, slide the slider at the top of the screen from Inactive to Active. Anzo loads the graphmart.
Now that the data is loaded into AnzoGraph, it is available to view in dashboards. Proceed to the next step to explore the sample data with Hi-Res Analytics.
- On the Graphmart screen for the sample movie data graphmart, click the Create Dashboard button. Anzo opens the Hi-Res Analytics application and displays the New Dashboard dialog box:
- On the New Dashboard dialog box,specify a Title for the dashboard and add an optional Description. Leave the default Graphmart dashboard value in the Type field. For example:
- Click OK to create the dashboard. The new dashboard appears as a new tab on the screen and contains a sub-tab titled What can I do next?. This tab acts as a wizard to guide you through the initial dashboard creation.
Anzo populates the Graphmart and Data Layers panels with the sample movie data graphmart and default data layer for the graphmart.
- To define the type of data from the graphmart to display on the dashboard, click the Specify the type of data you would like to see link on the What can I do next tab. The Select Data Types dialog box appears and displays the available data types. Since the sample data set has one type or class of data, only IMDB-Movie-Data is listed. The value in parentheses shows the total number of instances of that type.
- Click IMDB-Movie-Data to select it, and then click OK to add the data type to the dashboard.
- In the main Hi-Res Analytics toolbar, click the Dashboard button and select Save to save the dashboard.
To start to explore the data and see what values exist for the properties in the IMDB-Movie-Data class, it can help to add a filter to the dashboard. Filters reveal the values associated with properties. Learning more about the values enables you to start making decisions about what properties to group on, for example, what properties have relationships, and what results you want to visualize.
- To create a filter, click the Add facets to filter the data link on the What can I do next tab. Anzo displays the Create Filter dialog box:
- In the Create Filter dialog box, click the Fields field and browse the available properties. For this tutorial, select Year, and then click Close to close the Fields drop-down list.
- Click the Filter drop-down list and browse the available filter types. For this tutorial, select Single Select List. Anzo adds the filter type to the Create Filter dialog box.
- At the bottom of the screen under Format, click the Type drop-down list and select No format. Removing formatting from the filter ensures that commas are not displayed for the Year values. The image below shows the completed dialog box.
- Click OK to close the dialog box and add the new filter to the dashboard. Anzo adds the Year filter to the left panel. The new filter will enable you to narrow the scope of the data when it is visualized.
Once you have a good understanding of the values and relationships that exist in the data set, you can experiment with lenses and decide on the most appropriate way to display the data. Creating a Table lens is a quick way to view the data that you filtered.
- To create a lens, click the Select or create visualizations of your data link on the What can I do next tab. Anzo displays the Lens Selection dialog box. Since lenses have not been created previously, the list of lenses is empty:
- In the Create Lens dialog box, click the Create a new lens link at the bottom of the screen. Anzo displays the Create Lens dialog box:
- In the list of lens types, select the Table lens and then click Next. Anzo displays the General Information dialog box:
- Type a Title for the lens, and then click Finish. Anzo adds the lens as a new sub-tab on the dashboard and opens the Table Designer:
- In the Designer, click the Auto-generate columns icon () to add all available columns to the table.
Since the sample data set includes only 12 columns, it is not overwhelming to view all columns at once. For larger data sets, you might want to be more selective when adding columns to table lenses.
- Click Save. The table lens displays on the dashboard and populates with data. For example:
You can reduce the number of results that are displayed by clicking a year value in the Year filter.
- In the Hi-Res Analytics main toolbar, click the Dashboard button and select Save to save the changes. Remember to save dashboards periodically.
Now that you can view a summary of the data in a table, it can help you determine how to further narrow or expand the results by adding, changing, or removing filters. In addition, you can experiment by adding other lenses to the dashboard to find the ideal way to display the data to answer the questions that you have. For example, the table includes a Revenue column. It might be interesting to see which movies made the most revenue. The next steps guide you through creating a chart lens to display movie revenue.
- In the Hi-Res Analytics main toolbar, click Lenses and select New. The Create Lens dialog box opens.
- Select Chart and click Next. Anzo displays the General Information dialog box.
- Specify a Title and add an optional Description for the new lens. For example:
- Click Finish. Anzo displays the Chart Designer, which enables you to configure the type of chart to create, such as column, pie, or line, specify the data that will populate the chart, and customize details such as the chart's theme and fonts, legend, title, and tooltip formats.
- For this tutorial, accept the default Chart Type of Column, and click Chart Data to view the Chart Data screen and configure the data to display.
- Click the Group field and select the Title property from the drop-down list. The Group field sets the values for the x-axis on the column chart. Anzo also populates the Group label with the Title property. Click Close to close the drop-down list.
- Click the Value field and select the Revenue (Millions) property from the drop-down list. The Value field sets the y-axis values. Click Close to close the drop-down list.
- Click Save. Anzo adds the new chart lens to the dashboard. To narrow the results that are displayed and filter on one year at a time, click a year in the Year filter. For example, by clicking 2006 in the filter, the chart lists the revenue for titles that were released in 2006 and shows that "Pirates of the Caribbean: Dead Man's Chest" had the most revenue for that year:
You can hover the pointer over a bar in the chart to view details.
- If you want to add more data to the chart, such as to compare revenue versus rating for each title, follow these steps:
- Open the chart designer by clicking the Designer button above the lens tabs.
- In the Designer, click Chart Data.
- On the Chart Data screen, click the Add a series link at the bottom of the screen. A new series, named Series 2, is added to the list.
- For Series 2, click the Group field and select the Title property from the drop-down list. Then click Close to close the list.
- Click the Value field and select the Rating property from the drop-down list. Close the list.
- Click Save to save the chart changes. The chart updates to display the Rating values in addition to the Revenue. For example:
For more information about working with dashboards, see Analyzing Data with Hi-Res Analytics in the Anzo Deployment and User Guide.
- Open the chart designer by clicking the Designer button above the lens tabs.
This section briefly reviews the artifacts that were created as part of the new sample solution and provides references to more details about each of the artifacts.
During the process of automatically ingesting the sample movie data from the supplied CSV file, Anzo generated the following components:
- A Model, which describes the structure and type of data that the movie data set contains. For more information and instructions for viewing the derived model, see Modeling Data.
- A Mapping, which is input to the ETL job and maps the source data to the target elements defined in the data model. For more information and instructions for viewing the generated mapping, see Working with Mappings.
- A Dataset Pipeline, which contains the ETL job that generates the target data set. For more information and instructions for viewing the generated pipeline, see Configuring Pipelines.