Getting Started


Overview

DNIF is a proactive data analytics platform that can ingest, parse and enrich large volumes of data each day and bounce back with actionable intelligence using complex compute models and rules.

DNIF will help you:

  • Ingest
  • Parse
  • Enrich
  • Index
  • Search
  • Process
  • Analyse your data

After you’ve installed Docker and DNIF on your host system, you can start using the DNIF web console in just 4 simple steps.

To start using the web console, you need to:

  1. Logging in to the DNIF-Analytics Console.
  2. Connecting and Executing your First Query.
  3. Using the DNIF-Analytics Console.

To get you started with using the web console, as an example, you will be shown how you can fetch and aggregate logs generated by the top 10 devices within the last 12 hours.

The output of this search will then be visualised using a widget and, eventually, a dashboard.

The above search will be performed , by executing a query in DNIF Query Language on sample web server logs, loaded using a file called weblogs.csv, which you can download from here.

The subsequent article will get you started with the DNIF web console using the above example with data from weblogs.csv.

Logging in to the DNIF Web Console

To login to the console:

DNIF Web Console
  • Enter the following details:
Element/Field Mandatory Description
Email ID Yes Enter your registered email id.
Password Yes Enter the password.
OTP Yes Enter the OTP generated by Google Authenticator plugin.

NOTE: OTP field becomes visible only after we click GO after entering our registered email id. DNIF uses two step authentication for added security. This method prevents attack using stale data.

An email with Sign in to Web Console as the subject line provides you with the required Auth Secret key to get started with the Google Authenticator application.

Authentication code is received in email
  • Click Login.

    You will be logged in to the DNIF console.

Search page initially before setting up connection

Connecting from your workstation to the DNIF Server (A10 Server)

Before you start playing with the DNIF Web Console, it is important for you to connect your workstation and A10 Server.

NOTE: In distributed deployments where the Adapters, Datastores and Correlators are deployed individually, you need to connect your workstation with the datastore.

To establish this connection:

1. From the MANAGEMENT menu, click CONNECTIONS. A CONNECTIONS page is displayed.

Snippet of Connections page

2. In the Source Address field, enter the IP address of the DNIF server.

3. From the Action tab, click Save.

The connection is saved.

NOTE: Saving the connection will not guarantee workstation and A10 server connection. You need to refresh and link the saved connection. The steps for the same are given below.

4. Click Refresh icon to see whether your saved connection setting is displayed or not.

5. To link your workstation and the A10 server, click connect icon

A new window with a hello message is displayed and the link icon changes it color from red to blue.

This implies that: - the connection is successful between your workstation and A10 server - the connection is encrypted using a self signed SSL certificate.

Creating an Event Store

Now that you have created a connection, you can start uploading your files to query them.

To do this:

1. To upload a file , from the menu bar, click SEARCH. You will be navigated to the SEARCH screen.

Click on the OPTIONS menu

2. Click click on the hamburger icon and select EVENT STORE from the drop-down menu.

Select Event STORE option from OPTIONS menu

The CREATE EVENT STORE window is displayed.

EVENT STORE side panel view

NOTE: In this Getting Started article, we will be uploading the weblogs.csv file to create an event store.

3. Enter the required fields.

Element/Field Mandatory Description
Store name Yes Enter the required event store name.
Store type Yes Select the required store type from the drop down menu.
Type of passcode Yes Select the required store type from the drop down menu.

Store Type comprises of following options:

  • Create an empty store - This option can be used to create an empty event store.
  • Create and upload - This option can be used when you wish to upload a particular file and create an event store based on it.
  • Read from file - This option can be used when you wish to create an event store based on an already existing file.

4. Click Save. An event store is created, saved and added successfully.

SEARCH page in DNIF Web Console

5. From the MANAGEMENT menu, click EVENT STORES.

Check whether your STORE was created

You must be able to see your newly created event store.

EVENT STORES section displays all the created events.

Required STORE as listed

NOTE: A newly created event store takes some time to be displayed in the EVENT STORES menu. Please refresh the screen and wait for few minutes.

Retrieving data using DNIF Query Language (DQL)

After creating an event store, you can start fetching your data using DQL.

To do so:

1. Click the SEARCH menu.

SEARCH interface to slice and dice data

A search window is displayed.

2. In the SEARCH box, type a simple fetch query and press enter.

_fetch * from weblogs limit 100

This query will return the last 100 web logs.
The query result is displayed as shown in the image given below:

Data within the STORE

Using the DNIF Web Console

You can create multiple widgets and dashboards in order to visualise the analysed data. To do this, you need to:

  • Create a package
  • Create a widget
  • Create a dashboard

Creating a Package

Packages are used to encapsulate multiple dashboards inside a single box (package).

NOTE: To create a dashboard, you need to have a package.

To create a new package:

1. From the MANAGEMENT menu, click REPOSITORY.

REPOSITORY view within DNIF Web console

This page displays all the created packages.

View all the existing PACKAGES

2. Click Click on ADD icon to create PACKAGE

Encapsulate workflow and threat hunting modules in single entity

The REPOSITORY/PACKAGE/ page is displayed.

Fields to fillup before creating a PACKAGE

3. On the REPOSITORY/PACKAGE/ page, complete the following details:

Element/Field Mandatory Description
Name Yes Enter the required package name.
Description Yes Enter the required package description.
Image Yes Enter the required package image (100x100 px).
Visibility Yes Select the required package visibility.

Visibility of the package can be any one of the following:

  • Public: - The package will be visible to everyone once you upload it on the DNIF server.
  • Private: - The package will not be visible to anyone even if you upload it on the DNIF server.
  • Share - Add the people with whom you want to share the package. The added people can only see the package and not modify it.
  • Admins - Add the people with whom you want to share the package and give admin rights. The added people can edit the package too.

4. Click Save.

Your package is created and saved. Package is added successfully under the REPOSITORY menu.

Newly created PACKAGE as shown in PACKAGE section

Creating a Widget

After you are done creating a package, the next step is to create a widget. Widgets are used to create a graph based on the result retrieved by any particular query.

To create a widget:

1. Click on the SEARCH menu.

A SEARCH page is displayed.

SEARCH page can be utilised to investigate threats

2. In the SEARCH box, type a simple aggregate query as given below and press Enter.

_fetch * from weblogs where $Duration = 12h group count_unique $SrcIP limit 10

This query will fetch required entries within the web logs file, which were generated within the last 12 hours from the last 10 unique IP addresses.

3. To create a widget for the search results generated in step 2, click Click on MORE icon

Create contextualised widget

4. From the drop-down menu, click CREATE WIDGET. The REPOSITORY/WIDGET/ page is displayed.

Visualise your data by creating WIDGETS
Select the appropriate chart to visualise

5. On the REPOSITORY/WIDGET/ page, complete the following details:

Element/Field Mandatory Description
Name Yes Name of the widget.
Query Yes Query for which the widget is to be made.
Chart Type Yes Select the required chart you want to display. You can select from a range of charts like pie chart, line chart, bar chart, maps, pyramid, table chart, and so on.
Primary key Yes Enter the required primary key to uniquely identify each record.
Secondary key Yes Enter the required secondary key.
Value Yes Enter the required value.
Context detail Yes Add the required context menu.
  • Various chart types available are as follows:
    • Pie Chart
    • Line Chart
    • Vertical Bar Chart
    • Horizontal Bar Chart
    • Map
    • GeoMap
    • Solid Gauge
    • Pyramid
    • Pyramid
    • Conditional Display
    • Spline Chart
    • Text Chart
    • Table Chart

6. Click Save.

This will create and save your widget.

WIDGET create successfully

Creating a Dashboard

You can use a dashboard to plot widgets for several Key Performance Indicators (KPIs). It helps provide a different perspective on your data and facilitates comparison between different types of KPIs.

A dashboard enables you to simply drag and drop widgets and resize them according to your requirements.

NOTE: Before creating a dashboard, make sure you have created a package and widget.

To create a dashboard:

1. Click on the DASHBOARD menu.

A DASHBOARD page is displayed.

A single DASHBOARD view to monitor all Security operations

2. Click ADD icon

Select PACKAGE in which DASHBOARD needs to be created

A SELECT PACKAGE window is displayed.

3. Select the required package from the drop down menu and click Proceed.

Select relevant PACKAGE

4. Enter the name of dashboard and click Save.

Save your DASHBOARD

The dashboard is added and displayed under the DASHBOARD menu.

Newly created DASHBOARD as listed

5. Open the created dashboard.

Open the newly created DASHBOARD

Since the dashboard is new, no widgets are displayed.

6. Click ADD icon

A Widget Settings page is displayed.

Enter fill up the fields within the WIDGET settings

7. Enter the following details:

Element/Field Mandatory Description
Select Widget Yes Select the required widget.
Select a scope Yes Select the required scope.
Show labels Yes Specify whether you want data labels to render on the chart.
Primary key Yes Enter the required primary key to uniquely identify each record.
Secondary key Yes Enter the required primary key to uniquely identify each record.
Polling interval(minutes) Yes Enter the polling interval after which you want your dashboard to be updated automatically.
Context detail Yes Add the required context menu.

8. Click Add to dashboard.

The dashboard is saved successfully.

ADD WIDGET to your DASHBOARD

9. Click SAVE button to add a new widget to the dashboard.

This will save the widget to your dashboard for future use.

Getting started with DNIF the open big data analytics platform

NOTE: A single dashboard can have multiple widgets.