Running Federated Search from Splunk

📘

TL;DR Using Query from Splunk

1. Confirm that your administrator has installed the Query app in Splunk.
2. Go to the Splunk Dashboard or Search box and start your search with a pipe '|' in front of the queryai command, like this:

3. Tailor your search with parameters and operators as described below. Run | queryai help=allfor sub-options to understand search syntax, available connectors, attribute namings, etc.

Search Command Syntax

The queryai command accepts from one to three parameters.

Syntax:

| queryai search=”<search conditions>” events="<list of events>" connectors="<list of connectors>" timeout=<seconds>

Basic example (with minimum parameters set):

| queryai search=”<field> <operator> <value>”

Specific example with more parameters set:

| queryai search="ip = 172.16.16.10" events="detection_finding, file_activity" connectors="S3, elastic, sentinel" timeout=60

Parameters

search

A mandatory parameter that carries your search conditions and boolean logic.

connectors

An optional comma-separated parameter identifying the connector aliases that you want to include in your search (e.g., MS Defender, S3, etc.). Without this parameter, all connectors will be queried.

Note: Your connectors are unique to your configured data sources and you can get that list that using | queryai help=connectors. Query platform administrators can configure additional connectors from the 'Connections' page in Query's Console at https://go.query.ai/.

exclude-connectors

An optional comma-separated parameter identifying the connector aliases that you want to exclude in your search (e.g., MS Defender, S3, etc.).

events

An optional comma-separated parameter identifying the event types that you want to include in your search (e.g.: file_activity, detection_finding.). Without this parameter, all events will be queried.

Note: You can see the list of supported event classes at https://schema.query.ai/events.html

exclude_events

An optional comma-separated parameter identifying the event types that you want to exclude in your search (e.g.: file_activity, detection_finding).

timeout

An optional numeric parameter that lets you set the maximum time in seconds you are willing to wait for results. Note that setting this timeout may lead to incomplete results, as only the results obtained within this timeframe will be displayed.

Help, Examples, and Attribute and Connector Namings

Here are common help options available from within Splunk search:

• Understand all the help sub-options: | queryai help=all
• Help on search syntax, along with examples: | queryai help=search
• Get names of connectors configured in your environment: | queryai help=connectors
• Get searchable list of Entities, Events, and Objects: | queryai help=[entities|events|objects]
• Get schema details of a given Event/Object: | queryai help=<event/object name>

Operators

You can tailor your queryai searches using the following operators.

Search Types

Searches can be considered to be one of two types:

• Entity searches are very simple and allow you to retrieve a wealth of data related to a single Entity. Entity searches may include connectors and timeout parameters, and/or a single condition (i.e., Equals, Contains, Starts With, or Ends With), to limit your results.
• Power searches are searches for specific Objects and Events, tailored with conditions and parameters to return precisely what you’re looking for. Power searches are not difficult, but when performing them, you should refer to the Query Data Model (QDM) for maximum effectiveness.

We’ll explain both of these types of searches below, starting with a basic Entity search.

Entity Search

Query makes it extremely simple to search across diverse, distributed sets of data to find common resources such as hostnames, IP addresses, file hashes, and more. These common resource types are called Entities.

Adding a condition (for example "=") to your Entity search retrieves matching results only.

You don’t need to learn anything about data models to perform highly-effective Entity searches like these:

| queryai search="ip = 8.8.8.8"
| queryai search="file_hash = b5045d802394f4560280a7404af69263"

The list of searchable Entities includes:

- account_id  
- account_name  
- command_line  
- country  
- credential_id  
- cve_id  
- cwe_id  
- email  
- file_hash  
- file_name  
- group_id  
- group_name  
- hostname  
- ip  
- mac_address  
- port  
- process_id  
- process_name  
- resource_id  
- subnet  
- url  
- user_agent  
- user_id  
- username

If you would like to understand Entities in more detail, you can read our guide explaining normalization of QDM Entities here.

Power Searches: Events and Objects

To search for Events and Objects, all of the same syntax and conditions apply as for Entity searches. However, you should also familiarize yourself with relevant sections of the Query Data Model (QDM) so that you can specify the searchable Object and Event types that you are looking for.

You can browse QDM Events and Objects at https://schema.query.ai.

Top-level Events and Objects in the QDM with a checkbox in the “Searchable” column can be specified in your searches. For example, the image below shows that the Object "Actor" (with name "actor") is a searchable Object.

Objects and Events always have attributes, which could be simple data types like a string, or a complex type, i.e., another object. These attributes can be examined further by clicking on them.

When you perform a search for the specific Event or Object attributes that you are interested in, use a dot to separate hierarchically-related attributes, for example:

| queryai search="user.email_addr = [email protected]"
| queryai search="actor.authorizations.decision=denied"
| queryai search="user.groups IN (admin, superadmin)"
| queryai search="vulnerability_finding.confidence_id = HIGH"

You could also do a wildcard search (with no attributes) to get a list of all results for that Event, as shown below:

| queryai search="scan_activity = *"

Above star searches are only supported on scalar attributes and on Events classes. They are not supported on Entities and Objects. For Objects, or Objects contained in Events, you can instead do a star search on one of their underlying scalar attributes.

In your search syntax, use the QDM Event or Object Name from the QDM schema rather than the display name: for example, use the lower snake-case "datastore_activity" rather than "Datastore Activity" (the latter is meant for display readability).

Event Search

QDM Events are cybersecurity events (or activities) of interest, such as Authentication Events.

The list of searchable Events include:

- account_change
- api_activity
- application_lifecycle
- authentication
- authorize_session
- compliance_finding
- config_state
- data_security_finding
- datastore_activity
- detection_finding
- dns_activity
- email_activity
- email_delivery_activity
- file_activity
- file_hosting
- http_activity
- incident_finding
- module_activity
- network_activity
- osint_inventory_info
- patch_state
- process_activity
- security_finding (deprecated)
- user_inventory
- vulnerability_finding
- web_resources_activity

To understand QDM Events in detail, you can see the guide explaining normalization of QDM Events here.

Object Search

QDM Objects are cybersecurity objects of interest, such as the User Object. As opposed to Events, you can think of Objects as things.

The list of searchable Objects includes:

- actor
- agent
- data_security
- device
- domain_info
- domain_intelligence
- file
- file_intelligence
- ip_intelligence
- url_intelligence
- user

To understand QDM Objects in detail, you can see the guide explaining normalization of QDM Objects here.

Search Examples

Equality

| queryai search="ip = 1.1.1.1"  
| queryai search="hostname = My-MacBookPro"

Not Equals

| queryai search="user.devices.domain != xyzdomain"
| queryai search="email != *@xyz.com"

Starts With

| queryai search="hostname = mac*"  
| queryai search="user.name = sam*"

Ends With

| queryai search="hostname = *mac"  
| queryai search="user.name = *anand"

Contains

| queryai search="hostname = *mac*"  
| queryai search="user.name = *ana*"

Show all (using * wildcard)

Search for all Events of the QDM Event class by using the wildcard '*'.

Or use '*' in an attribute to get the Objects/Events that have a value for that attribute.

| queryai search="authentication = *"
| queryai search="authentication.logon_type_id = *"
| queryai search="user.email_addr = *"

Explicit AND-ing multiple conditions

| queryai search="user.name=ana AND [email protected]"

Implicit AND-ing multiple conditions

| queryai search="user.name=ana [email protected]"

OR-ing multiple conditions

| queryai search="user.name=ana OR [email protected]"

In

| queryai search="user.name IN (ana, ava, 'abc xyz', 'abc pqr')"

Negating condition(s) using NOT

| queryai search="NOT (user.name=ana OR [email protected])"
| queryai search="NOT user.name IN (ana, ava, 'abc xyz', 'abc pqr')"

Noteworthy Search Scenarios: Tips for Success

Left associativity

In the absence of parentheses, the default order of evaluation is left to right. So this search:

| queryai search="user.type=System OR [email protected] AND user.name=ava"

would be executed as:

| queryai search="(user.type=System OR [email protected]) AND user.name=ava"

---

Using parentheses to prioritize associativity

| queryai search="user.type=System OR ([email protected] AND user.name=ava)"

Nesting of parentheses is not supported. However, you can use multiple parentheses to associate separate condition sections.

---

Case Sensitivity

• Entities, Events, Objects, and their attribute names are case-sensitive.
• Values are generally case-insensitive. (An exception is enum attribute values, covered in the "Conditions on enum attributes" section below.)

---

Checking for null or not-null values

You can search for an attribute that has a non-empty value. For example, you can search for users with a valid value for email_addr like this:

| queryai search="user.email_addr = *"

You can also search for an attribute that does not have a value, i.e., checking for nulls. Here is an example search for users for whom email_addr is not set:

| queryai search="NOT user.email_addr = *"

---

dot(.) notation for attribute hierarchy

Use the dot(.) notation to identify hierarchical attributes in any event or object. For example:

| queryai search="user.devices.domain=xyzdomain"

---

Conditions on enum attributes

Enum attributes hold a value from a specific set. For example, incident_finding.impact_id can only have one of these values: LOW, MEDIUM, HIGH, or CRITICAL. (Refer to the QDM documentation to see all enum attributes and their allowed values.)

Here are some scenarios to be aware of with enum attributes:

• You can only use equality or inequality conditions, i.e., you cannot use ‘starts with,’ ‘contains,’ or ‘ends with’ style conditions.
• Use uppercase for enum attribute value letters: Rather than ‘Informational,’ use ‘INFORMATIONAL.’
• Although the QDM documentation shows both values and an internal numeric code for enum values, only use the uppercase style values, not the internal codes. For example, rather than a scan_activity.severity_id value of "1," use "INFORMATIONAL."

---

Handling spaces, special characters, or reserved keywords

Values containing reserved keywords (like AND, OR, NOT, IN), whitespaces, or special characters (like dots) should be enclosed inside single or double quotes. For example:

| queryai search="user.email_addr='[email protected]'"

However, 'Starts With,' 'Ends With,' and 'Contains' operators do not support quoted values.

---

Conditions on certain attribute types

Operators like 'Starts With,' 'Ends With,' and 'Contains' are only supported with attributes that hold strings. So, for example, the following Entities don't support those operations:

• ip
• mac_address
• file_hash

---

Connection-specific condition limitations

Depending upon the connection, certain conditions may not be supported. For example, threat intelligence platforms like VirusTotal don't support 'Starts With,' 'Ends With,' and 'Contains' operators on url.

---

Don't put search conditions across different entity types

By design, Entity Searches are simple. Each Entity Search supports one condition on a single entity type - but not across multiple entity types. Also, you can't do an unconstrained entity=* search.

For example, you cannot do:

• ip=*
• ip=1.1.1.1 OR ip=1.2.3.4

Instead, you can search for multiple values within the same entity type through the use of IN / Starts With / Ends With. You could also use an ORed Object / Event search.

Examples:

• ip IN (1.1.1.1, 1.2.3.4)
• hostname="DATACENTER1*"
• device.ip=1.1.1.1 OR device.ip=1.2.3.4

No AND-ing across multiple event/object types

When AND-ing multiple conditions, you must use the same Object and Event types between the conditions. For example, this is valid, because the conditions are on the same Object: "user":

| queryai search=user.name = abc AND user.type!=System

But the below example is not valid, because the conditions are on different Objects: "user" and "device":

| queryai search=user.name = abc AND device.ip=1.1.1.1

If you mean to use a relationship-oriented condition, you can alternatively do this (which is valid because the conditions are on the same Object: "user"):

| queryai search=user.name = abc AND user.devices.ip=1.1.1.1

It is valid to do OR-ing across multiple event/object types. For example, this is valid if you want to see these two unrelated Objects together.:

| queryai search=user.name = abc OR device.ip=1.1.1.1

---

Connection-specific Error Handling With Partial Results

Searches performed from the Query platform initiate multiple distributed parallel queries run via relevant connectors into your data sources. There could be connection-specific errors or warnings that cause a subset of data connections to provide only partial results. In that scenario, the app will still show you the available results, and will also show you the specific warnings and errors from the connections that didn't provide results.

---

Extracting data from stringified JSON hierarchy

Leveraging the JSON viewer:

OCSF is a hierarchical schema, i.e. it contains sub-objects further containing sub-fields (which could be objects too having their own sub-fields). The _raw field contains the native platform's raw data (mapped from OCSF's raw_data) and Splunk only shows its JSON viewer for data in _raw field. To apply the JSON viewer to another field, you can assign its value into the _raw field:

... | eval _raw=xyzfield

If you want to preserve the original _raw, then you can do:

... | rename _raw AS original_raw | eval _raw=xyzfield

Extracting Interesting Fields

By default, Splunk extracts Interesting Fields only from the top-level fields in the JSON hierarchy. Splunk stringifies to flatten the hierarchical data, and that could make it hard for the analyst to visually scan sub-fields. However, all it takes is using the spath command to extract the relevant sub-field(s) of interest:

... | spath input=xyzfield

If you need to extract from another stringified field or from another subfield of xyzfield, you can continue chaining spath to extract further from multiple fields. For example:

... | spath input=xyzfield | spath input=pqrfield

Extracting only specific sub-paths

Instead of extracting all sub-fields, if you care only about a certain field, you can extract using:

... | spath input=xyz path=pqr output=xyz_pqr

Note that In above example, we also specified an output field name prefix to be added to the subfields, so that we can easily distinguish.

Keeping only desired fields in the output

You can chain further on top of the above examples and do... | fields a, b, c, d to only keep fields you desire to keep - a, b, c, d in our example. You can also alternatively remove undesired field(s) by adding a dash in front: ... | fields -xyz, -pqr