Gmail Messages API

📘

TL;DR

To integrate Google Workspace with Query:

• Activate the required Google Workspace APIs from a Google Cloud Project: Gmail API.
• Create a Service Account, assign it Google Workspace admin & domain-wide delegation and create a JSON key.
• Activate email delegation for any users you want to audit emails for via the Query app, this can only be done via email address.
• Configure the Gmail API Connector.
• Use Query to search for information on Users and Emails for usage by IR, Investigations, Threat Hunting, Audit, and Security Operations teams.

Overview

Google Workspace (formerly known as G-Suite) is a cloud-based productivity suite that offers tools for collaboration, communication, and organization, such as Gmail, Google Drive, Google Calendar, and Google Meet. It enables businesses to streamline workflows, manage files securely, and collaborate in real-time across various locations and devices.

Security and IT personnel use Google Workspace to manage user access, enforce security policies, and monitor activity, helping protect organizational data through advanced security features like two-step verification, data loss prevention (DLP), and mobile device management (MDM). Google Workspace's centralized admin console provides IT teams with powerful tools for user management, security insights, and policy enforcement to safeguard company information.

As of 1 DEC 2024, Query integrates with Google Workspace across three distinct Connectors associated with data surfaced by the following APIs: Directory API, Reports API and the Gmail Messages API. This separation of Connectors by API allows customers to assign different Service Accounts, different delegations, and limit overall access to certain APIs to different Teams and Organizations within Query. The following capabilities are supported from the aforementioned APIs. Refer to the API-specific documentation in Query for more information on how to configure specific connectivity.

📘

On Gmail Messages API Authorization

If a Domain-wide admin delegate or specific delegate is not specified, Query's Gmail Connector will establish the authenticated session using your Service Account and pass the target email addresses as the session context. This will allow you to still search for specific emails even if you do not use delegation.

All federated searches have their searches and results expressed in the terms of the Query Data Model (QDM), which is based on the Open Cybersecurity Schema Framework (OCSF). Each API source is normalized into a specific QDM/OCSF Event Class to standardize and normalize the data for increased situational awareness, ease of aggregation of filtering, and easy pivoting.

Executing federated searches with Query allows you to pull all relevant data for your search criteria back in a parallelized, normalized, and standardized format without ever moving or duplicating the data into another data repository. For instance, searching for an Email Address Entity in Query will allow you to pull all relevant email messages for the user(s). Likewise, you can request Emails that fulfill other conditions, or none at all to sample email data within Gmail. When used alongside the Google Workspace Connector, you can correlate the activity against User and Device configurations as well as Google Drive, Admin, and Authentication logs.

❗️

Message Searching Limitations

Every search that starts with the Gmail Messages API requires a starting mailbox -- either specified by an Email Address or User UID -- while this can be done via Entity-based searches by Email, it does not allow fine-grained control for more specific search parameters such as asking for a specific from or to email.

To attempt to fulfill searches such as these, we will retrieve batches of 400 messages by using the Delegated Administrator parameter where possible until a match is found.

Prerequisites

To use any of the three Google Workspace Connectors, you must first ensure that you are on a paid tier of Google Workspace (e.g., Business Starter, Business Standard, Enterprise) otherwise you will not have access to the APIs. Additionally, you should have Super Admin, Admin, or a similarly privileged role for your Google Workspace Admin portal to be able to complete or verify completeness of the following steps.

1. Firstly, ensure you have access to a Google Cloud project with an Owner role, or have permissions to enable Google Workspace APIs. For more information see the Create a Google Cloud project section of the Google Workspace Guide.
2. Next, enable the Admin SDK API and Gmail API by navigating to the menu in your Google Cloud project and navigating to More Products --> Google Workspace --> Product Library and toggling the APIs. For more information see the Enable Google Workspace APIs section of the Google Workspace Guide. You can alternatively enable these APIs with the gcloud CLI.

gcloud services enable gmail.googleapis.com

3. Next, follow the steps detailed in the Service account credentials section of the the Create access credential section of the Google Workspace Guide. At a high level you will create an IAM Service Account, assign it as an Admin, toggle domain-wide delegation, and generate a JSON Service Account Key.

4. Optionally, for better audibility and regulatory compliance requirements, consider either Delegating administrator privileges in Gmail or enable delegation for Users and create a specific Connector per user you intend to audit or investigate message content for.

5. To retrieve the Customer ID for your Google Workspace tenant from the Admin console navigate to Account --> Account settings and copy the value underneath the Profile header as shown below (FIG. 1).

   

Configuring the Gmail API Connector

To configure a Connector for Google Workspace in the Query Federated Search platform, you must first verify completeness of all prerequisites, have access to the Google IAM Service Account JSON Key, and have the Customer ID of your Google Workspace tenant. For the Gmail API specific Connector, you must provide the email address for a Domain-wide delegated administrator, the delegated admin for a specific mailbox, or the specific email address of a user you wish to investigate or audit.

1. Navigate to the Connectors page, select Add Connectors, and selectGmail API from the Email Security and Communications category as shown below (FIG. 2). Optionally, you can search for the Connector using the search bar at the top of the Add Connectors drawer.

   

2. In the Configure Connector section, add the following detail as shown below (FIG. 3):

   1. Connector Alias Name: The human-readable name you want to give to this connector, you can name it whatever you want, but you can use this to delineate across different APIs or Instances, especially if you're a part of a Managed Security Services Provider (MSSP). You can also use it to keep different Delegated Administrators identified, if used.

   2. GCP Service Account JSON Key Contents: The pasted JSON contents of your GCP Service Account that has Domain-wide delegation enabled for your Google Workspace tenant that you created in Step 3 of the Prerequisites section.

   3. Delegated Administrator Email: Optional parameter. Enter in a Delegated Administrator or a specific mailbox delegate you wish to use to provide authentication for searching.

      

3. Select Save to save and activate the Connector.

4. Select Test Connection from the bottom-right of the connection pane to ensure that your Service Account credentials are valid, that it is setup correctly, and that Query can successfully dispatch a search against the various APIs.

You will now see Crowdstrike Falcon added as an available Connector within the Query Search and Query Summary Insights UI.

Querying Gmail API Connectors

Within the Query Search UI, all Connectors are enabled by default. To check that your specified Connector(s) for Crowdstrike Falcon are enabled, navigate to the Email Security and Communications section of the Selected Connectors dropdown and ensure that your specified Gmail API Connector(s) are are selected (denoted by a checkbox) before running your searches as shown below (FIG. 4).

To learn more about searching with the Gmail API Connector, refer to the subsections below.

Entity-based Search

To conduct an Entity-based search, toggle the Select an Entity dropdown menu in the Federated Search console and ensure the Entity radio button is selected (this is the default option) as shown below (FIG. 5). The typeahead menu searches from the beginning of the Entity, so typing "U", will bring Username, User Agent, and User ID to the top of the list for instance.

The Gmail API is a static schema Connector which means that all normalization and search translation is completely defined by the Query team. Refer back to the Introduction section and refer to the table to learn which Entities map against which Gmail API methods. The User ID or Email Address that you search with will be the targets of the Messages search.

Since Entity searches are broadly applicable searches, you will get different types of Results across different Connectors. While the Gmail API will just return Messages for the subjects that you search for, other information that maps to Email Address in other Connectors can be brought back. This includes devices and user record entires from the Google Workspace Directory API, logs and events for them from the Google Workspace Reports API, and other Connectors that may have details by Email Address such as Authentication logs from Okta or Amazon Security Lake (for AWS CloudTrail) may also return.

After selecting an Entity, most allow you to specify an Operator. This allows you to perform simple equality searches or to perform more generalized searches using contains, starts with, or ends with Operators. These operators, however, are not able to be natively implemented.

Multiple values can be searched at the same time which are automatically OR'ed and parallelized where possible as different search requests. Additionally, case sensitivity can be toggled for the entire search result as well. An example of a multi-value "Hostname contains" search is shown below (FIG. 6).

Event-based Search

Event-based searches allow you to broadly search across the entirety of results from a downstream API, or, search for very specific results based on filters. Refer back to the Introduction section and refer to the table to learn which Events map to the Gmail API.

As of 30 NOV 2024, the Gmail Messages API does not support this style of search, a subject must always be provided.

Resources

https://developers.google.com/gmail/api/guides

Troubleshooting Tips

• Ensure that the Gmail API is enabled in your Google Cloud project.
• Ensure that your Service Account has domain-wide delegation in your Google Workspace tenant.
• Ensure that you are searching for a valid user in your tenant.
• Ensure that the delegated administrator you provide, is actually setup as such.
• Attempt to increase the time range of your searches if you are not getting the expected results back.

If you have exhausted the above Troubleshooting list, please contact your designated Query Sales Engineer or Customer Success Manager. If you are using a free tenant, please contact Query Customer Success via the Support email in the Help section, or via Intercom within your tenant.