2ms (Too many secrets)

Too many secrets (2ms) is an open source CLI tool, powered by Checkmarx, that enables you to identify sensitive data such as secrets, authentication keys and passwords that are stored in your system in unencrypted text. This tool supports scanning of internal communication platforms (Slack, Discord), content management (Confluence, Paligo) and source code storage locations (Git repo, local directory).
This application is written in Go language and is based on the framework provided by gitleaks.

The tool checks the content using a series of rules that are designed to identify a wide range of sensitive items such as AWS access token, Bitbucket Client ID, GitHub PAT etc. For a complete list of rules, see docs/list-of-rules.md.

Additionally, the tool incorporates a scoring system based on the Common Vulnerability Scoring System (CVSS) to help prioritize remediation efforts.

Installation

The following sections explain how to install 2ms using the following methods:

• Homebrew (macOS/Linux)
• Download and Install Precompiled Binaries
• Compile from Source
• Run From Docker Container
• CI/CD Integrations

Homebrew (macOS/Linux)

You can now install 2ms directly via Homebrew:

brew install 2ms

Once installed, verify the installation with:

2ms --version

Download and Install Precompiled Binaries

You can download 2ms precompiled binaries for amd64 architecture from our releases page. The following links can be used to download the "latest" version, for each supported OS.

• Download for Windows
• Download for MacOS
• Download for Linux

Install Globally

Install 2ms globally on your local machine by placing the compiled binary on your path. For example, on Linux you can place 2ms binary in /usr/local/bin/ or create a symbolic link.

Example:

cd /opt
mkdir 2ms
cd 2ms
wget https://github.com/checkmarx/2ms/releases/latest/download/linux-amd64.zip
unzip linux-amd64.zip
sudo ln -s /opt/2ms/2ms /usr/local/bin/2ms

Compile from source

You can compile the project from its source using the following commands:

git clone https://github.com/checkmarx/2ms.git
cd 2ms
go build -o dist/2ms main.go
./dist/2ms

Run From Docker Container

We publish container image releases of 2ms to checkmarx/2ms . To run 2ms from a docker container use the following command:

docker run checkmarx/2ms

Mounting a Local Directory

You can also mount a local directory by using the-v flag with the following syntax -v <local-dir-path>:<container-dir-path>

Example:

docker run -v /home/user/workspace/git-repo:/repo checkmarx/2ms git /repo

• For git command, you need to mount your git repository to /repo inside the container

CI/CD Integrations

GitHub Actions

The following is a template for creating a GitHub Action that runs 2ms from a Docker image to scan your GitHub repo.
Note: Make sure that in the actions/checkout step you access the full history by setting the depth as follows fetch-depth: 0

name: Pipeline Example With 2MS

on:
  pull_request:
    workflow_dispatch:
    push:
      branches: [main]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
        with:
          # Required for 2ms to have visibility to all commit history
          fetch-depth: 0

      # ...

      - name: Run 2ms Scan
        run: docker run -v $(pwd):/repo checkmarx/2ms:2.8.1 git /repo

• This example uses version to 2.8.1 of 2ms. Make sure to check for the latest version.
• 💡 Take a look at 2ms GitHub Actions pipeline as 2ms scans itself using 2ms.

Azure DevOps Pipeline

To use 2ms in Azure DevOps Pipeline, create a new pipeline (see this tutorial for getting started with Azure DevOps Pipelines). Then, use the following template to create a yml file azure-pipelines.yml to run 2ms:

trigger:
  - master

pool:
  vmImage: ubuntu-latest

steps:
  - script: docker run -v $(pwd):/repo checkmarx/2ms:2.8.1 git /repo
    displayName: Run 2ms

• This example uses version to 2.8.1 of 2ms. Make sure to check for the latest version.

Running 2ms Scans

Overview

2ms has dedicated commands for scanning each of the supported platforms. To run a scan, you need to enter the command for the platform that you are scanning, along with all of the arguments that are relevant for that platform. The scan command arguments are used for authentication as well as to provide details about the locations that will be scanned. These arguments differ for each platform. In addition, you can add global flags to customize the scan configuration.
The fundamental structure of a scan command is:

2ms <scan command> [scan command arguments] [global flags]

Scan command arguments and global flags can be passed either as flags in the scan command or via a config file.

Command Line Help

We've built the 2ms command line interface to be as self-descriptive as possible. This is the help message that is shown when you execute 2ms without args:

2ms Secrets Detection: A tool to detect secrets in public websites and communication services.

Usage:
  2ms [command]

Scan Commands
  confluence  Scan Confluence Cloud
  discord     Scan Discord server
  filesystem  Scan local folder
  git         Scan local Git repository
  paligo      Scan Paligo instance
  slack       Scan Slack team

Additional Commands:
  completion  Generate the autocompletion script for the specified shell
  help        Help about any command
  rules       List all rules

Flags:
      --add-special-rule strings      special (non-default) rules to apply.
                                      This list is not affected by the --rule and --ignore-rule flags.
      --allowed-values strings        allowed secrets values to ignore
      --config string                 config file path
  -h, --help                          help for 2ms
      --ignore-on-exit ignoreOnExit   defines which kind of non-zero exits code should be ignored
                                      accepts: all, results, errors, none
                                      example: if 'results' is set, only engine errors will make 2ms exit code different from 0 (default none)
      --ignore-result strings         ignore specific result by id
      --ignore-rule strings           ignore rules by name or tag
      --log-level string              log level (trace, debug, info, warn, error, fatal, none) (default "info")
      --max-target-megabytes int      files larger than this will be skipped.
                                      Omit or set to 0 to disable this check.
      --regex stringArray             custom regexes to apply to the scan, must be valid Go regex
      --report-path strings           path to generate report files. The output format will be determined by the file extension (.json, .yaml, .sarif)
      --rule strings                  select rules by name or tag to apply to this scan
      --stdout-format string          stdout output format, available formats are: json, yaml, sarif (default "yaml")
      --validate                      trigger additional validation to check if discovered secrets are valid or invalid
  -v, --version                       version for 2ms

Use "2ms [command] --help" for more information about a command.

Configuration File

You can pass --config [path to config file] argument to specify a configuration file. The configuration file format can be YAML or JSON.

Example:

log-level: info

regex:
  - password\=

report-path:
  - ./report.yaml
  - ./report.json
  - ./report.sarif

paligo:
  instance: your-instance
  username: your-username

Hybrid Configuration Mode

You can pass a combination of command line arguments and a configuration file. In this case, the 2ms merges the values from the file and the explicit arguments.

.2ms.yml config file:

ignore-result:
  - b0a735b7b0a2bc6fb1cd69824a9afd26f0f7ebc8
  - 51c76691792d9f6efe8af1c89c678386349f48a9
  - 81318f7350a4c42987d78c99eacba2c5028636cc
  - 8ea22c1e010836b9b0ee84e14609b574c9965c3c

Command: The --spaces flag is provided in the CLI command (outside of config file):

Example:

docker run -v $(pwd)/.2ms.yml:/app/.2ms.yml checkmarx/2ms \
    confluence --url https://checkmarx.atlassian.net/wiki \
    --spaces secrets --config /app/.2ms.yml

Scan Commands

The following sections describe the arguments used for scanning each of the supported platforms.

Confluence

This command is used to scan a Confluence instance.

2ms confluence <URL> [flags]

Flag	Type	Default	Description
--space-keys	string list	(all)	Comma-separated list of space keys to scan.
--space-ids	string list	(all)	Comma-separated list of space IDs to scan.
--page-ids	string list	(all)	Comma-separated list of page IDs to scan.
--history	bool	false	Also scan all versions of each page (page history).
--username	string		Confluence username/email (used for HTTP Basic Auth).
--token-type	string		Token type for Confluence API. Accepted values: api-token, scoped-api-token.
--token-value	string		The API token value. Required when --token-type is set.

Authentication

• To scan private spaces, provide --username, --token-type and --token-value (API token).
• How to create a Confluence API token: https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/

Examples

• Scan all public pages (no auth):

  2ms confluence https://<company id>.atlassian.net/wiki

• Scan private pages with an api token (requires auth):

  2ms confluence https://<company id>.atlassian.net/wiki --username <USERNAME> --token-type api-token --token-value <API_TOKEN>

• Scan private pages with a scoped api token (requires auth):

  2ms confluence https://<company id>.atlassian.net/wiki --username <USERNAME> --token-type scoped-api-token --token-value <API_TOKEN>

• Scan specific spaces by key:

  2ms confluence https://<company id>.atlassian.net/wiki --space-keys Key1,Key2

• Scan specific spaces by ID:

  2ms confluence https://<company id>.atlassian.net/wiki --space-ids 1234567890,9876543210

• Scan specific pages by ID:

  2ms confluence https://<company id>.atlassian.net/wiki --page-ids 11223344556,99887766554

• Include page history (all revisions):

  2ms confluence https://<company id>.atlassian.net/wiki --history

Paligo

Scans Paligo content management system instance.

Flag	Value	Default	Description
--instance	string	-	Instance name
--token	string	-	API token for authentication
--username	string	-	Paligo username
--folder	string	scanning all instance's folders	Folder ID
--auth	string	-	Base64 auth header encoded username:password

Discord

Scans Discord chat application history.

Flag	Value	Default	Description
--token	string	-	Discord token
--channel	strings	all channels will be scanned	Discord channel IDs to scan
--messages-count	int	0 = all messages will be scanned	The number of messages to scan
--duration	duration	14 days	The time interval to scan from the current time. For example, 24h for 24 hours or 336h0m0s for 14 days
--server	strings	-	Discord servers IDs to scan

How to get a Discord token.

Example:

2ms discord --token <YOUR_TOKEN> --server 1097814317077897307 --duration 9999h

Slack

Scans Slack chat application history.

Flag	Value	Default	Description
--token	string	-	Slack token
--channel	strings	all channels will be scanned	Slack channel IDs to scan
--messages-count	int	0 = all messages will be scanned	The number of messages to scan
--duration	duration	14 days	The time interval to scan from the current time. For example, 24h for 24 hours or 336h0m0s for 14 days
--team	string	-	Slack team name or ID

Git Repository

Scans a local git repository

2ms git <Git Repo Local Path> [flags]

Flag	Value	Default	Description
--all-branches	-	false - only current checked in branch	scan all branches
--depth	int	no limit	limit the number of historical commits to scan from HEAD
--base-commit	string	-	base commit to scan commits between base and HEAD

For example

git clone https://github.com/my-account/my-repo.git
cd my-repo
2ms git .

Local Directory

Scans a local repository

2ms filesystem --path PATH [flags]

Flag	Value	Default	Description
--path	string	-	Local directory path
--project-name	string	-	Project name to differentiate between filesystem scans
--ignore-pattern	strings	-	Patterns to ignore

Example:

2ms filesystem --path .

Global Flags

The following table describes the global flags that can be used together with any of the scan commands.

Flag	Type	Default	Description
--add-special-rule	string		Add special (non-default) rules to apply. This list is not affected by the --rule and --ignore-rule flags. SEE BELOW
--config	string		Path to the config file
-h, --help	string		Help for 2ms commands
--ignore-on-exit		None	Defines which kind of non-zero exits code should be ignored. Options are: all, results, errors, none. For example, if 'results' is set, only engine errors will make 2ms exit code different from 0.
--ignore-result	strings		Ignore specific result by ID
--ignore-rule	strings		Ignore rules by name or tag.
--log-level	string	info	Type of log to return. Options are: trace, debug, info, warn, error, fatal, none
--max-target-megabytes	int		Files larger than than the specified threshold will be skipped. Omit or set to 0 to disable this check.
--regex	stringArray		Custom regexes to apply to the scan. Must be valid Go regex.
--report-path	strings		Path to generate report files. The output format will be determined by the file extension (.json, .yaml, .sarif)
--rule	strings		Select rules by name or tag to apply to this scan.
--stdout-format	string	yaml	Stdout output format, available formats are: json, yaml, sarif
--validate			Trigger additional validation to check if discovered secrets are valid or invalid. SEE BELOW
-v, --version			Version of 2ms that is running.

Validity Check

Adding the --validate flag checks the validity of the secrets found. For example, if a Github token is found, it will check if the token is valid by making a request to the Github API. We will use the least intrusive method possible to check the validity of the secret.

The list of services that support the Validity Check feature can be found in the List of Rules document.

The result of the validation can be:

• valid - The secret is valid
• invalid - The secret is invalid
• unknown - We failed to check, or we are not checking the validity of the secret at all

If the --validate flag is not provided, the validation field will be omitted from the output, or its value will be an empty string.

Note: The validity check also impacts the score field. If the flag is not provided, the validity is assumed to be "unknown" in the score formula.

Special Rules

Special rules are rules that are configured in 2ms but are not run as part of the default ruleset, usually because they are too noisy or too specific. You can use the --add-special-rule flag to add special rules by rule ID.

For example:

2ms git . --add-special-rule hardcoded-password

List of Special Rules

Rule ID	Description
hardcoded-password	Detects strings that are assigned to variables that contain the word password, access, key, etc.

Custom Regex Rules

You may specify one or more custom regex rules with the optional argument --regex. The value provided will be parsed as a regular expression and will be matched against the target items.

my-file.txt

password=1234567
username=admin

2ms filesystem --path . --regex username= --regex password=

Contributing

2ms is extendable with the concept of plugins. We designed it like this so anyone can easily contribute, improve and extend 2ms. Read more about contributing in our CONTRIBUTING.md file.

Contact

Want to report a problem or suggest an idea for improvement? Create an Issue, create a Discussion thread, or Join our Discord Server (seek for #2ms channel)

This project was made and maintained by Checkmarx with ❤️