Amazon Security Lake

📘

TL;DR

To integrate Amazon Security Lake with Query:

• Setup Amazon Security Lake for your AWS Account or AWS Organization in one or more Regions
• Select a source-specific (e.g., Amazon VPC Flow Logs, AWS Security Hub, Wiz CNAPP) and custom-source (coming soon!) Connector, stage with the appropriate information, and receive an External ID
• Deploy an IAM Role with External ID that provides permissions to interact with Amazon Athena, AWS Glue, Amazon S3, and AWS KMS (optionally)
• Grant the IAM Role SELECT and DESCRIBE permissions within AWS Lake Formation to specific Databases and Tables in your Amazon Security Lake data lake
• Confirm connectivity into your source with the Test Connection functionality in your Connector
• Use Query Search to surface information about IP Addresses, Hostnames, Domains, and Events within your Amazon Security Lake deployment.

Overview

Amazon Security Lake is an Amazon Web Services (AWS) Cloud service that "automatically centralizes security data from AWS environments, SaaS providers, on premises, and cloud sources into a purpose-built data lake stored in your account. With Security Lake, you can get a more complete understanding of your security data across your entire organization. You can also improve the protection of your workloads, applications, and data. Security Lake has adopted the Open Cybersecurity Schema Framework (OCSF), an open standard. With OCSF support, the service normalizes and combines security data from AWS and a broad range of enterprise security data sources".

Security Lake supports AWS first-party sources as well as a wide variety of Partner-provided data sources, additionally, customers can create their own custom sources in the lake. Query supports all Security Lake first-party integrations with AWS Services such as Amazon Virtual Private Cloud (VPC) Flow Logs and Amazon Route 53 Resolver Query logs and select Partner integrations that are treated as first-party connections such as Wiz.io.

In the future, Query will support auto-mapping any supported OCSF Event Class into the Query Data Model (QDM) which is based on OCSF v1.2.0-dev with certain additive Extensions to support broader use cases. Query integrates with Amazon Security Lake by interfacing with Amazon Athena as the distributed query engine and does not require Subscriber Access to the lake nor does Query persist or otherwise copy your data from Amazon Security Lake. Additionally, Query does not store or duplicate any data, it is best suited for operational and ad-hoc use cases to support Security Operations (Incident Response, Investigations, Threat Hunting), Red Team targeting operations, and security engineering and compliance use cases.

As of 7 APR 2024 the Query Federated Search Platform integrates with the following sources in Amazon Security Lake for both Security Lake V1 (OCSF 1.0.0-rc2 & AWS Glue/Hive table format) and Security Lake V2 (OCSF 1.1.0 & Apache Iceberg table format).

• Amazon Route 53 Resolver Query Logs
• Amazon Virtual Private Cloud (VPC) Flow Logs
• AWS CloudTrail Management Events
• AWS CloudTrail S3 Data Events
• AWS CloudTrail Lambda Execution Events
• Amazon Elastic Kubernetes Services (EKS) Audit Logs
• AWS Security Hub Findings
• Wiz.io CNAPP findings

See the OCSF Source Identification documentation to learn more about how AWS maps into various OCSF Event Classes.

Using these following integrates users of Query Federated Search can search for observables within these schemas to locate and pivot from Domains, Hostnames, IP Addresses, Resource IDs, and Usernames. With Query Federated Search, all results are normalized, deduplicated, correlated, collated and enriched such that an entity-based search would surface similar data points across all of your onboarded resources.

As an example, an Analyst investigating EDR findings could query by an AWS Cloud-hosted server with an EDR sensor by its IP Address and surface all related CloudTrail, VPC Flow Log, and Route 53 Resolver Query traffic.

Prerequisites

Before using the Query Federated Search Connector for Amazon Security Lake, ensure that you have followed the steps within the Amazon Security Lake Getting Started guide.

Additionally, you must create an Amazon S3 bucket and configure Amazon Athena to write results to it, the S3 bucket itself can be referenced using the Bucket Name or the Amazon Athena Workgroup Name for the Connector.

Lastly, if you do not have the AWS CLI installed and/or configured, refer here. Ensure you have permissions to create and modify IAM Roles and Policies and add AWS Lake Formation principal permissions.

• iam:CreateRole
• iam:CreatePolicy
• iam:AttachRolePolicy
• lakeformation:GrantPermissions

For the best experience for your Security Operations and other supported functions that use Amazon Security Lake, strongly consider onboarding all of the AWS Accounts within your AWS Organization and additionally consider using "Rollup Regions" to consolidate and centralize cross-Region data into a single Account and single Region.

However, it is worth noting that the Query Federated Search Platform can create multiple different Connectors per Amazon Security Lake Source if you would rather not use Rollup Regions. Additionally, Query contains the credentials necessary for querying Amazon Security Lake with each unique Connector, enabling you to onboard lakes in multiple different AWS Accounts and Regions regardless of the principal AWS Organization.

Configure an Amazon Security Lake Connector - Static Schema

Amazon Security Lake Connectors within the Query Federated Search Platform can take the form of a static schema or a dynamic schema (Coming Soon!) platform configuration. Static schemas are platforms in which the Query team pre-configures the type of data normalization that happens and a dynamic schema platform gives the user control for mapping and normalizing data into the Query Data Model. For more reading on this topic see the Dynamic Schema Configuration and the Normalization and the Query Data Model sections, respectively.

NOTE Amazon Security Lake Connectors for static schema platforms are also known as "first party" Connectors for Amazon Security Lake and contain AWS-managed Sources in their own dedicated Connector such as that for Amazon Virtual Private Cloud (VPC) Flow Logs and select Custom-sources such as Wiz.io CNAPP Security Findings.

To enable connectivity into your AWS Account, the Query Federated Search platform uses AWS IAM Roles with a read-only, randomly generated UUID as the ExternalId per unique connector configured. This random ExternalId is generated after entering in all relevant details, so you will need to determine what you will name your AWS IAM Role ahead of time. As a matter of best practice, this mechanism is intended to create a single Policy, a Single Role, and to be used with a single Connector with its own distinct Lake Formation permissions.

In the tasking order, you will:

1. Provide required details to a selected Amazon Security Lake Connector, for the purpose of this documentation, CloudTrail Management Events will be used as the example.
2. Create an IAM Policy with the generated Identity Policy snippet
3. Create an IAM Role with the generated Trust Policy snippet, attach the IAM Policy
4. Grant the IAM Role appropriate Lake Formation permissions to the specific Security Lake Database and Table
5. Test Connection and enable the Connector

Proceed to the next sections to learn how to activate an Amazon Security Lake Connector and start searching.

Pre-configure an Amazon Security Lake Connector

Use the following steps to create a new Query Federated Search Connector for Amazon Security Lake, in this example we will onboard AWS CloudTrail Management Events, however the same steps can be used for any of the static schema or "first party" Security Lake Connector within the Query Federated Search Platform. Be sure to change the required details such as the Table Name and using the proper IAM Role.

1. Navigate to the Connections page, select Add Connections, and selectAmazon Security Lake - AWS CloudTrail Management Events from the Cloud Infrastructure & Security category, optionally type "Security Lake" in the search bar as shown below (FIG. 1)

   

2. In the General tab, add the following details listed below the screenshot (FIG. 2).

   

   1. Connection Alias Name: The human-readable name you want to give to this connector, you can name it whatever you want, such as naming it after the source data in each Table you query.

   2. AWS Account ID: The same AWS Account ID where your Athena Workgroup, Target and Results Amazon S3 Buckets are located.

   3. Role Name: The name of the AWS IAM Role that delegates access to the Query Federated Search AWS Account. Remember, you will create this Role AFTER pre-configuration, ensure you use the same Role Name when you do so.

   4. External ID: This value will be generated after you Save the Connector.

   5. Security Lake S3 Bucket Name: The name of the Amazon S3 bucket created by Amazon Security Lake.

   6. Catalog Name: The name of the AWS Glue Data Catalog in your AWS Account, this should be AwsDataCatalog in most cases but can also be default.

   7. Database Name: The name of the AWS Glue Database which contains the AWS Glue Table or Amazon Athena View that you query.

   8. Table/View Name: The name of the AWS Glue Table or Athena View that corresponds to the Security Lake Source you are trying to query, for instance this should be amazon_security_lake_table_us_east_2_cloud_trail_mgmt_1_0

   9. Data Source Provider: Leave default (e.g., CloudTrail Management)

   10. Results S3 Bucket Name: The name of the Amazon S3 bucket that you configured to write Amazon Athena query results into.

   11. Amazon Athena Workgroup Name (OPTIONAL): In lieu of supplying a value for Results S3 Bucket Name and KMS Key you can supply a custom Amazon Athena Workgroup that defines these values.

   12. Results Encryption Option: A dropdown selection for the type of encryption used, either Amazon S3-managed Keys (SSE-S3), your own AWS KMS CMK or an external encryption key used for Customer-Side Encryption (CSE). You must use the SAME type of encryption in both areas, if you do not specify any encryption in Amazon Athena, leave this as Amazon S3-managed keys (SSE-S3).

   13. KMS Key (OPTIONAL): The AWS Resource Name (ARN) or the GUID of the AWS KMS CMK used for SSE-CMK or SSE-CSE, if used. If you use SSE-S3 encryption leave this option blank.

   14. AWS Region: The same AWS Region where your Athena Workgroup, Security Lake and Results Amazon S3 Buckets are located.

3. Select Save from the bottom-right of the connection pane as shown below (FIG. 3). This ensures that all supplied information is saved and allows the Query backend to generate IAM Policy JSON snippets and an External ID unique to this Connector.

   

4. Scroll up copy the AWS IAM policy snippets as shown below (FIG. 4) for both the Trust Policy (for your IAM Role) and Identity Policy (for your IAM Policy) and proceed to the next section.

Create AWS IAM Policy for Amazon Security Lake Connector

In this section you will use the supplied Identity Policy from the previous section to create an AWS IAM Policy. While only a specific Glue Database and Table combination is specified, your entire Security Lake and Amazon Athena Results S3 Buckets have permissions granted for them. If you want to limit the Role to a specific prefix (folder/directory) path in Amazon S3 for either bucket, ensure you modify the JSON snippet as necessary.

If you encrypt your data in S3 using AWS KMS Customer Managed Keys (CMKs) and/or encrypt your Amazon Athena query results, you must also add the following permissions to this IAM Policy.

• kms:Decrypt
• kms:GenerateDataKey
• kms:Encrypt

The following steps use an Amazon Linux 2023 AWS CloudShell environment and the AWS CLI, adapt these instructions to your specific operating system or deployment strategy (e.g., AWS CloudFormation, Terraform, AWS Console, etc.).

1. Create an environment variable that contains the content of your Identity Policy as shown below, ensure you replace the section { "YOUR_POLICY_HERE } with your actual policy!

POLICY_JSON=$(cat <<EOF
{ "YOUR_POLICY_HERE }
EOF
)

2. Create the AWS IAM Policy with the following command, ensure you change the policy-name value for whatever name you wish to grant the policy.

aws iam create-policy \
    --policy-name "QueryFederatedSearchSecLakePolicy" \
    --description "Grants the Query Federated Search Platform Glue, Athena, and S3 permissions to Security Lake resources" \
    --policy-document "$POLICY_JSON"

3. (OPTIONAL STEP) if you use AWS KMS CMKs to encrypt the Amazon Athena query results and/or to encrypt the data within your Security Lake S3 Bucket, add the following policy into the created AWS IAM Policy using the AWS Console. Ensure you replace the values of $AWS_REGION, $AWS_ACCOUNT_ID and $$YOUR_KEY_ID_HERE with your proper AWS Region, AWS Account ID, and KMS Key ID respectively.

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Sid": "kmsPermissions",
			"Effect": "Allow",
			"Action": [
				"kms:Decrypt",
				"kms:Encrypt",
				"kms:GenerateDataKey"
			],
			"Resource": "arn:aws:kms:$AWS_REGION:$AWS_ACCOUNT_ID:key/$YOUR_KEY_ID_HERE"
		}
	]
}

In the next section you will create an AWS IAM Role, attach this Policy to it, and optionally modify you AWS KMS Key Policy if you use a KMS CMK. Ensure you copy the value of the IAM Policy ARN as you will need it in the next section!

Create AWS IAM Role & attach IAM Policy

In this section you will use the supplied Trust Policy from the previous section to create an AWS IAM Role which trusts the Query Production Account and delegates access to your Glue, Athena, S3 (and optionally KMS) resources. The ExternalId that is generated is to prevent a Confused Deputy flaw within the role assumption logic.

We do however realize this can be cumbersome to maintain multiple specific IAM Roles per-Connector despite it being a best practice. To that end, if this matches with your own internal identity governance (and other) requirements, you can specify multiple ExternalId entires in one IAM Trust Policy.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::822484525064:root"
        ]
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": [
            "foo",
            "bar",
            "69420",
            "vampirevampire"
            ]
        }
      }
    }
  ]
}

If you encrypt your data in S3 using AWS KMS Customer Managed Keys (CMKs) and/or encrypt your Amazon Athena query results, you must also add the following permissions to your AWS KMS Key Policy and trust the Principal (the IAM Role) you will create.

• kms:Decrypt
• kms:GenerateDataKey
• kms:Encrypt

The following steps use an Amazon Linux 2023 AWS CloudShell environment and the AWS CLI, adapt these instructions to your specific operating system or deployment strategy (e.g., AWS CloudFormation, Terraform, AWS Console, etc.).

1. Create an environment variable that contains the content of your Trust Policy as shown below, ensure you replace the section { "YOUR_TRUST_POLICY_HERE } with your actual policy!

TRUST_POLICY_JSON=$(cat <<EOF
{ "YOUR_TRUST_POLICY_HERE }
EOF
)

2. Create the AWS IAM Role with the following command, ensure you change the role-name value for whatever name you pre-configured the Connector with, you can change the name in the Connector but not the AWS IAM Role!

aws iam create-role \
    --role-name "QueryFederatedSearchForSecLake" \
    --description "Delegates access to Amazon Athena and related services to the Query Federated Search Platform" \
    --assume-role-policy-document "$TRUST_POLICY_JSON"

3. Attach the IAM Policy you created in the previous section with your IAM Role with the following command, ensure you replace the values for role-name and policy-arn for your actual values.

aws iam attach-role-policy \
    --role-name "QueryFederatedSearchForSecLake" \
    --policy-arn "arn:aws:iam::ACCOUNT_ID:policy/QueryFederatedSearchSecLakePolicy"

4. (OPTIONAL STEP) if you use AWS KMS CMKs to encrypt the Amazon Athena query results and/or to encrypt the data within your Security Lake S3 Bucket you must add the following Statement to your AWS KMS Key Policy. Replace $AWS_ACCOUNT_ID and $ROLE_NAME with your AWS Account ID and the created IAM Role name, respectively.

{
    "Sid": "queryFederatedSearchPermissions",
    "Effect": "Allow",
    "Principal": {
        "AWS": "${IAMRoleArn}"
    },
    "Action": [
        "kms:Decrypt",
        "kms:Encrypt",
        "kms:GenerateDataKey"
    ],
    "Resource": "arn:aws:iam::$AWS_ACCOUNT_ID:role/$ROLE_NAME"
}

In the next section you will grant this IAM Role SELECT and DESCRIBE Lake Formation permissions to be able to gain read access to your Database, Tables (and optionally, any future Views) created for Amazon Security Lake.

Grant AWS Lake Formation Principal Permissions to your AWS IAM Role

AWS Lake Formation helps you centrally govern, secure, and globally share data for analytics and machine learning. With Lake Formation, you can manage fine-grained access control for your data lake data on Amazon Simple Storage Service (Amazon S3) and its metadata in AWS Glue Data Catalog. Amazon Security Lake uses Lake Formation to control fine-grained access to your security data lake's Databases, Tables, and the specific Actions (e.g., SELECT, CREATE, DROP) that can be taken by certain principals.

You can use the AWS Console to provide the IAM Role you created with access to every Action and Table within your Amazon Security Lake deployment, or, you can use more Bash scripts and the AWS CLI to provide "exactly one" minimum necessary permissions per IAM Role that corresponds to a specific Database and Table. Like other resource-based and session-based policies in the AWS Cloud, you will require Identity-based permissions in addition to Principal Permissions from Lake Formation to successfully integrate the Query Federated Search Platform with your data in Amazon Security Lake.

Grant AWS Lake Formation permissions using the AWS Console

Ensure that you set your AWS Region to the same Region where your Security Lake rollup Region is located.

1. In the AWS Console navigate to AWS Lake Formation -> Data lake permissions and select Grant as shown below (FIG. 5)

   

2. In the Principals section, select IAM users and roles and in the dropdown search bar labeled IAM users and roles enter the IAM Role you created previously in this guide.

3. In the LF-Tags or catalog resources section, select Named Data Catalog resources and choose the Amazon Security Lake created Database(s) as shown below (FIG. 6), the below screenshot also encompasses Step 2.

   

4. In the Tables dropdown either select the specific AWS Glue Table you setup permissions for in your IAM Role, or, select All tables to grant access to every single Amazon Security Lake table within the Database as shown below (FIG. 7).

   

5. (OPTIONAL STEP) if you have created Amazon Athena Views based on your Security Lake data lake Tables, select them in the Views dropdown to choose All views to account for future created Views. This option only works if you do not specify Tables.

6. Finally, in the Table permissions section choose Select and Describe for both Table permissions and Grantable permissions and choose All data access in the Data permissions section and select Grant as shown below (FIG. 8).

   

You have now successfully AWS Lake Formation Principal Permissions to your AWS IAM Role used for the Query Federated Search Platform to access your Amazon Security Lake resources. In the next section you will finalize configuring the Connector for your Amazon Security Lake resources.

Grant AWS Lake Formation permissions using the AWS CLI

If you want to have "exactly one" minimum necessary permissions per Lake Formation Grant, use the following bash script to use the AWS CLI to create the Grant to a specific Database:Table Pair.

1. Create a Shell script with the following code snippet and save it to file (e.g., grant_lakeformation_permissions.sh)

#!/bin/bash

echo "Enter Database Name:"
read DATABASE_NAME
echo "Enter Table Name:"
read TABLE_NAME
echo "Enter AWS Account ID:"
read AWS_ACCOUNT_ID
echo "Enter Role Name:"
read ROLE_NAME

ROLE_ARN="arn:aws:iam::$AWS_ACCOUNT_ID:role/$ROLE_NAME"

aws lakeformation grant-permissions \
    --principal "{\"DataLakePrincipalIdentifier\": \"$ROLE_ARN\"}" \
    --resource "{ \"Table\": { \"DatabaseName\": \"$DATABASE_NAME\", \"Name\": \"$TABLE_NAME\" }}" \
    --permissions "SELECT" \
    --permissions-with-grant-option "SELECT"

echo "Granted SELECT permission to $ROLE_ARN for $DATABASE_NAME.$TABLE_NAME"

aws lakeformation grant-permissions \
    --principal "{\"DataLakePrincipalIdentifier\": \"$ROLE_ARN\"}" \
    --resource "{ \"Table\": { \"DatabaseName\": \"$DATABASE_NAME\", \"Name\": \"$TABLE_NAME\" }}" \
    --permissions "DESCRIBE" \
    --permissions-with-grant-option "DESCRIBE"

echo "Granted DESCRIBE permission to $ROLE_ARN for $DATABASE_NAME.$TABLE_NAME"

2. Make the script executable: chmod +x grant_lakeformation_permissions.sh.
3. Run the script and enter the following pieces of information when prompted: ./grant_lakeformation_permissions.sh
   1. Database Name: The name of the AWS Glue Database which contains your intended Table (or View) you intend to query.
   2. Table Name: the name of the Table you wish you query - this can be a table created using CREATE TABLE or a table created directly by AWS Glue Crawlers or another mechanism (e.g., AWS CLI, etc.)
   3. AWS Account ID: Your own AWS Account ID.
   4. Role Name The name of the AWS IAM Role you created in the previous section to attach the Policy to.

You have now successfully AWS Lake Formation Principal Permissions to your AWS IAM Role used for the Query Federated Search Platform to access your Amazon Security Lake resources. In the next section you will finalize configuring the Connector for your Amazon Security Lake resources.

Finalize configuring your Amazon Security Lake Connector

In this section you will verify all of your permissions, test that the connection works, and enable it so that you can dispatch searches to Amazon Security Lake from the Query Federated Search Platform.

1. Back in the Connections page, select the Amazon Security Lake Connector you configure and select Test Connection, you should receive the following message: The connection was tested successfully as shown below (FIG. 9).

   

2. Ensure that the Connector is still enabled by toggling the Connection Enabled to the right-hand position, though, this should not become disabled during the previous steps.

You will now see the name of your particular Amazon Athena Connector added as a new Platform within the Query Search UI, the "platform" term is synonymous with "connector" and "integration".

In the event that you have not achieved a successful Test Connection, refer to the Troubleshooting sub-section of Resources, below.

Querying from the Amazon Security Lake Connector

Within the Query Search UI, all Platforms (A.K.A Connectors) are enabled by default, to check that your specified Connector(s) for Amazon Security Lake are enabled, navigate to the Endpoint section of the Platforms dropdown and ensure that your specified Connector(s) for Amazon Security Lake are selected (denoted by a checkbox) before running your searches as shown below (FIG. 10).

The following Entities, Events and Objects are supported by Query for those data points. For more information about this terminology, refer to the Normalization and the Query Data Model (QDM) section of the docs or check out our QDM Schema website.

Unless otherwise noted, all Events and Entities (Observables) work for both Security Lake V1 (OCSF v1.0.0-rc2) and Security Lake V2 (OCSF v1.1.0) table formats.

AWS CloudTrail Management Events

Entities

• Username: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid, and actor.session.issuer within QDM and OCSF
  • E.g., Username EQ "[email protected]" or Username CONTAINS "contoso.org"
• Hostname: maps to src_endpoint.domain within QDM and OCSF
  • E.g., Hostname EQ "securitylake.amazonaws.com" or Hostname ENDSWITH "amazonaws.com"
• Resource ID: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid,actor.session.issuer, actor.user.credential_uid and resource[1].uid within QDM and OCSF
  • E.g., Resource ID EQ "aws:arn:::iam:user/myuser" or Resource ID Contains "role/"
• IP Address: maps to src_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"

Events

• Authentication
• Account Creation
• API Activity

AWS CloudTrail S3 Data Events

Entities

• Username: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid, and actor.session.issuer within QDM and OCSF
  • E.g., Username EQ "[email protected]" or Username CONTAINS "contoso.org"
• Hostname: maps to src_endpoint.domain within QDM and OCSF
  • E.g., Hostname EQ "securitylake.amazonaws.com" or Hostname ENDSWITH "amazonaws.com"
• Resource ID: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid,actor.session.issuer, actor.user.credential_uid and resource[1].uid within QDM and OCSF
  • E.g., Resource ID EQ "aws:arn:::iam:user/myuser" or Resource ID Contains "role/"
• IP Address: maps to src_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"

Events

• API Activity

AWS CloudTrail Lambda Execution Events

Entities

• Username: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid, and actor.session.issuer within QDM and OCSF
  • E.g., Username EQ "[email protected]" or Username CONTAINS "contoso.org"
• Hostname: maps to src_endpoint.domain within QDM and OCSF
  • E.g., Hostname EQ "securitylake.amazonaws.com" or Hostname ENDSWITH "amazonaws.com"
• Resource ID: maps to user.name,user.uid,user.uuid,actor.user.name,actor.user.uid,actor.user.uuid,actor.session.issuer, actor.user.credential_uid and resource[1].uid within QDM and OCSF
  • E.g., Resource ID EQ "aws:arn:::iam:user/myuser" or Resource ID Contains "role/"
• IP Address: maps to src_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"

Events

• API Activity

Amazon Route53 Resolver Query Logs

Entities

• Hostname: maps to query.hostname within QDM and OCSF
  • E.g., Hostname EQ "Amazonaws.com" OR Hostname CONTAINS "ntp"
• IP Address: maps to both src_endpoint.ip and dst_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"
• Resource ID: maps to src_endpoint.instance_uid, src_endpoint.vpc_uid, dst_endpoint.instance_uid, and dst_endpoint.interface_uid within QDM and OCSF
  • E.g., Resource ID EQ "i-123456abcdef" or Resource ID Contains "vpc-"

Events

• DNS Activity

Amazon Virtual Private Cloud (VPC) Flow Logs

Entities

• IP Address: Entity - maps to both src_endpoint.ip and dst_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"
• Resource ID: maps to src_endpoint.interface_uid,src_endpoint.vpc_uid,src_endpoint.instance_uid,src_endpoint.subnet_uid,dst_endpoint.interface_uid,dst_endpoint.vpc_uid,dst_endpoint.instance_uid,and dst_endpoint.subnet_uid within QDM and OCSF
  • E.g., Resource ID EQ "i-123456abcdef" or Resource ID Contains "vpc-" or Resource ID Contains "subnet-"

Events

• Network Activity

Amazon Security Hub Findings

Entities

• Resource ID: maps to finding.uid, finding_info.uid and resources[1].uid within QDM and OCSF
  • E.g., Resource ID EQ "i-123456abcdef" or Resource ID Contains "vpc-" or Resource ID Contains "arn:aws:us-east-1:"
• Process Name: maps to process.name within QDM and OCSF
  • E.g., Process Name EQ "ncat"
• File Name: maps to process.file.path and malware[1].path within QDM and OCSF
  • E.g., File Name Contains "/tmp/log/" or File Name EQ "msfvenom.py"

Events

• Security Finding (for OCSF v1.0.0-rc2)
• Detection Findings (for OCSF v1.1.0)
• Compliance Findings (for OCSF v1.1.0)
• Vulnerability Findings (for OCSF v1.1.0)

Amazon Elastic Kubernetes Service (EKS) Audit Logs

Entities

• Username: maps to user.name,user.uid, actor.user.name, actor.user.uid, and actor.session.issuer within QDM and OCSF
  • E.g., Username EQ "system:kube-scheduler" or Username CONTAINS "system"
• Resource ID: maps to user.name,user.uid, actor.user.name, actor.user.uid, actor.session.issuer, and resources[1].uid within QDM and OCSF
  • E.g., Username EQ "system:kube-scheduler" or Username CONTAINS "system"
• IP Address: maps to src_endpoint.ip within QDM and OCSF
  • E.g., IP Address EQ "10.100.1.0"
• URL: maps to http_request.url.path within QDM and OCSF
  • E.g., URL EQ "/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=5s" or URL CONTAINS "namespaces/airbyte"

Events

• API Activity

Wiz.io CNAPP Findings

Entities

• Resource ID: maps to finding.uid, finding_info.uid and resources[1].uid within QDM and OCSF
  • E.g., Resource ID EQ "i-123456abcdef" or Resource ID Contains "vpc-" or Resource ID Contains "arn:aws:us-east-1:"
• Process Name: maps to process.name within QDM and OCSF
  • E.g., Process Name EQ "ncat"
• File Name: maps to process.file.path and malware[1].path within QDM and OCSF
  • E.g., File Name Contains "/tmp/log/" or File Name EQ "msfvenom.py"

Events

• Security Finding (for OCSF v1.0.0-rc2)

Resources

Refer to the previous sections' hyperlinks for more information on specific resources, services, tools and concepts. For further help with creating tables and performance tuning, consider some of the resources below.

• What is Amazon Security Lake?
• Managing multiple accounts with AWS Organizations
• Managing multiple Regions
• Managing query access for Security Lake subscribers
• Security Lake queries
• SQL reference for Athena
• Working with views
• Performance tuning in Athena
• Top 10 Performance Tuning Tips for Amazon Athena

Troubleshooting Tips

• If you recently changed your permissions / Role in Query, log out and in again and clear your cache if you cannot Save or Test Connection.
• Verify that you were able to create the IAM Policy and Role and that they successfully attached.
• Ensure that Amazon Security Lake does not have reported Issues and that there is data in your AWS Glue Tables from Security Lake.
• Ensure that your Amazon Athena, Amazon S3, AWS KMS and Amazon Security Lake resources are all deployed in the same Region - despite Amazon S3 being a "global" resource - Athena and Security Lake do not respect this.
• Ensure that you entered the proper AWS Region for your Connector.
• Ensure that the Role Name you created matches the Role Name in your Connector
• Ensure that the AWS Account ID you entered in your Connector matches the AWS Account you have deployed Amazon Security Lake to, the Query Federated Search Platform does not use Subscriber Access.
• Ensure that other details such as the Glue Database, Glue Table, and the Amazon Security Lake S3 Bucket or Amazon Athena Results S3 Bucket names are correct and are aligned to the data source you're attempting to create.
• Verify that you gave the correct IAM Role Lake Formation SELECT and DESCRIBE permissions to the correct Database(s), Table(s) and/or Views.
• Verify that you are not using an AWS KMS CMK for either your Amazon Security Lake S3 Bucket or Amazon Athena Results S3 Buckets, if you are, refer back to the previous sections to update your AWS IAM Policy and your KMS Key Policies.
• Verify that your Organization, Organizational Unit(s), and/or AWS Account does not have a Service Control Policy (SCP) blocking external principals from sts:AssumeRole or other IAM actions related to Athena, Glue, and/or S3.

If you have exhausted the above Troubleshooting list, please contact your designated Query Sales Engineer or Customer Success Manager. If you are using a free tenant please contact Query Customer Success via the Support email in the Help section or via Intercom within your tenant.