Search API

The DNIF Search API lets you develop an application that can retrieve and display results with the desired DQL query programmatically. With this API, you can use Restful requests to get search results for multiple streams with desired start and end time.

Prerequisites


Ensure you have the following details before using the DNIF Search API

  • Console IP address
  • API Token

The API Token can be generated from the DNIF Console. To generate the token you can visit Manage Token.

API Overview


OperationDescriptionRest HTTP mapping
invokeReturns the requested id of the search for the DQL queryPOST
get statusReturns the status of task execution of requested idGET
get resultReturn the result of the search for the dql queryGET
How to use API?

Method Details

FieldDescription
HTTP MethodPOST
Response FormatJSON
Requires Authentication?Yes

Query Parameters

FieldValue
InvokeYou can retrieve the task_id for a particular search by sending an HTTP POST request to its URL. You pass in the details of the search request as the query body.
Call/invoke

URL

FieldValueDescription
URLhttps://<ipaddress>/<cluster_id>/wrk/api/job/invokeUrl should include the following parameters:

ipaddress: IP address of DNIF Console

cluster_id: Cluster id of the cluster you want to query data
Request Header

FieldValue
TokenSpecify the token generated from DNIF Console
Content-Typeapplication/json
Request Body

FieldValueDescription
query_timezone (string)Asia/KolkataSpecify the time_zone in which you are executing the query
scope_id (string)defaultSpecify the scope from where you want to retrieve data
job_typedqlSpecify the type of job you want to execute. Note: Only DQL is supported
job_executionon-demandSpecify the supported execution type Note: Only on_demand executions are supported.
query (string)_fetch * from event where $Stream=FIREWALL AND $StartTime=2021-09-14T17:42:00 AND $EndTime=2021-09-14T18:13:32 limit 10Specify the DQL (DNIF Query Language) query to retrieve data


All the above parameters fields are editable and are mandatory to invoke the call

Curl Command

curl -k --location --request POST 'https://<ipaddress>/<cluster_id>/wrk/api/job/invoke' \
--header 'Token: <token>' \
--header 'Content-Type: application/json' \
--data-raw '{
"query_timezone": "Asia/Kolkata",
"scope_id": "default",
"job_type": "dql",
"job_execution": "on-demand",
"query": "_fetch * from event where $Stream=FIREWALL AND $StartTime=2021-09-14T17:42:00 AND $EndTime=2021-09-14T18:13:32 limit 10"
}'
Response

Response contains all the details related to the invoked call, namely the status of the invoked call, the response results can be used to monitor progress, retrieve results, and/or delete it. Response will indicate whether the search is still being executed or it has completed or there was some error/failure.

Success
FieldValueDescription
Status_code200This status indicates that the call invoked was successfully completed and the search results are displayed as per the query.
StatusSuccess
Response Value 

{
"data": [
{
"id": "e646af00-66da-456f-a606-ca2b0a3dd19a-id-0",
"status": "PENDING"
}
],
"status": "success"
}
Error

FieldValueDescription
Status_code200This status indicates that the call invoked could not be completed due to some syntax error in the query parameter.
StatusError
Response Value 

{
"message": "query --> _fetch * from event limit | Error --> SyntaxError in _fetch query must end with limit / first / last and a valid number",
"status": "error"
}
Failure

FieldValueDescription
Status_code200This status indicates that the call invoked could not be completed due to some parameter error.
StatusFailed
Response Value 

{
"message": "Incorrect job_execution type",
"status": "failed"
}
Get Status

Method details

FieldDescription
HTTP MethodGET
Response FormatJSON
Requires Authentication?Yes

ParameterDescription/Value
/stateTo get the status of the invoke call, you need to send an HTTP GET request to its URL. You pass the id received from the invoke call as the query parameter.
URL

FieldValueDescription
URLhttps://<ipaddress>/<cluster_id>/wrk/api/dispatcher/task/state/<task_id>Url should include the following parameters:

ipaddress: IP address of DNIF Console

cluster_id: Cluster id of the cluster you want to query data
Header

FieldValue
TokenSpecify the token generated from DNIF Console
Query parameter

FieldValue
task_id:Specify the id received from the invoke call
Curl Command

 curl -k --location --request GET 'https://<ipaddress>/<cluster_id>/wrk/api/dispatcher/task/state/<task_id>' \
--header 'Token: <token>'
Response Status Code


Response status indicates whether a specific request has been successfully completed/Pending/failures.

Response CodeTask StageTask State
200EXECUTINGSTARTED
200EXECUTEDSUCCESS
200PENDINGQUEUED
200PENDINGQUERY_WORKERS_DOWN
200FAILUREFAILED
Response

FieldValueDescription
Status_code200This status indicates that the call invoked was successfully completed and the search results are displayed as per the query.
StatusSuccess
Response Value 

{
"status": "success",
"task_stage": "EXECUTED",
"task_state": "SUCCESS"
}

Check for the task_state result as success. When you get success as the result, execute the Get Result api to get the desired result.

Get Result

Method details

FieldDescription
HTTP MethodGET
Response FormatJSON
Requires Authentication?Yes
ParameterDescription/Value
/resultYou retrieve the result for the invoke call by sending an HTTP GET request to its URL. You pass the id received from the invoke call as the query parameter.
URL

FieldValueDescription
URLhttps://<ipaddress>/<cluster_id>/wrk/api/dispatcher/task/result/<task_id>?pagesize=<limit>&pageno=<page_no>Url should include the following parameters:

ipaddress: IP address of DNIF Console

cluster_id: Cluster id of the cluster you want to query data
Header

FieldValue
TokenSpecify the token generated from DNIF Console
Query parameter

FieldValue
task_idSpecify the id received from the invoke call
pagesizeSpecify the number of records in one call
pagenoSpecify the next page number (starts with 1)
Curl Command

curl -k --location --request GET 'https://<ipaddress>/<cluster_id>/wrk/api/dispatcher/task/result/<task_id>?pagesize=<size>&pageno=<number>' \ --header 'Token: <token>'
Response

FieldValueDescription
Status_code200This status indicates that the call invoked was successfully completed and the search results are displayed as per the query.
StatusSuccess
Response Value 

{"apply_hidden":true,"execution_time":"1.7275 sec","fields_list":["$CNAMTime","$System","$SourceName","$SourceType","$Stream","$Action","$SrcIP","$SrcPort","$DstIP","$DstPort","$Proto","$TXLen","$RXLen","$SystemTstamp","$EvtLen","$EventName","$SrcType","$IntelReference","$Status","$Type","$DstType","$IntelURL","$App","$DevSrcIP","$DstLOC","$DstCN","$DstASN","$DstISP","$SrcLOC","$SrcCN","$SrcASN","$SrcISP","$ExtractorID","$EStatus","$PStatus"],"result":[{"$Action":"PACKET_BLOCKED","$App":null,"$CNAMTime":1630074479965,"$DevSrcIP":"10.139.128.73","$DstASN":"396212","$DstCN":"US","$DstIP":"71.39.18.125","$DstISP":"AIMS-COMMUNITY-COLLEGE","$DstLOC":"[40.3851,-104.6759]","$DstPort":"53413","$DstType":"PUBLIC","$EStatus":"EAD","$EventName":null,"$EvtLen":500,"$ExtractorID":"127","$IntelReference":null,"$IntelURL":null,"$PStatus":"PAD","$Proto":"UDP","$RXLen":0,"$SourceName":"FORTIGATE","$SourceType":"FIREWALL","$SrcASN":"-","$SrcCN":"CN","$SrcIP":"116.5.35.83","$SrcISP":"-","$SrcLOC":"[23.1057,113.2023]","$SrcPort":"13031","$SrcType":"PUBLIC","$Status":"FAILED","$Stream":"FIREWALL","$System":"gotham-fortigate","$SystemTstamp":"2016-08-06","$TXLen":0,"$Type":null,"$Xhour":"2021082714","$Xminute":"202108271427"}],"source_stream":"FIREWALL","status":"success","task_status":"Task Executed Successfully","total_count":1}