Visualize Attack Paths in Production Environments With ThreatMapper

In this tutorial, learn how to install and use ThreatMapper in a Kubernetes environment.

The huge advances in 'Shift Left' processes makes it possible to deliver code to production that is secure and largely free from vulnerable dependencies. Among other things, these processes typically involve matching dependencies against public vulnerability lists from Mitre, Red Hat, Debian, and other projects.

'Shift Left' Scanning Alone Does Not Go Far Enough to Identify Production Vulnerabilities

Vulnerability lists don’t stand still. CVEs are published through the NVD at a rate of about 50 per day, so the risk of a new vulnerability being found in production is significant. Furthermore, third-party production components may not be scanned in a way that is as rigorous or up-to-date as you would like.

The Log4j 2 vulnerability, disclosed in December 2021, is an excellent case in point. As with all zero-day exploits, this was published without advanced warning, leaving enterprises scrambling to locate running software that contained the vulnerable Java component. (If you’d like to learn more, check out our blog post on how ThreatMapper helps enterprises find and fix their most exploitable log4j2 vulnerabilities.)

It quickly became clear that applications created by the enterprise were not the only area of concern; a great many third-party applications and tools also bundled that vulnerable component. 'Shift Left' scanning alone is not sufficient to identify supply-chain vulnerabilities.

Find Vulnerabilities in Production Applications and components

Deepfence’s open-source ThreatMapper project helps you address this gap. ThreatMapper scans your in-production workloads using current vulnerability feeds, and will find vulnerabilities that your 'Shift Left' scanning may have missed – there can be 1,000’s! ThreatMapper then reduces the list by assessing the risk-of-exploit, so you know what is most critical to fix first. It considers the attack vector, proximity to the network attack surface, and whether the affected workload has been seen to receive untrusted, external traffic.

ThreatMapper is composed of two components – an on-premises management console, and a series of sensors that you deploy to your production platforms. In this tutorial, we’ll show you how to install and use ThreatMapper in a Kubernetes environment. If you have any questions along the way, head on over to GitHub, or ask any questions you may have in our community Slack channel.

Getting Started: Overview

In this tutorial, we’ll go through the following steps:

1. Install the management console on a single Docker host; you’ll need a 4-vcpu, 16Gb host with docker and docker-compose

2. Create a first admin account on the new management console and obtain the API key

3. Install the ThreatMapper sensors on a production Kubernetes cluster, using the API key to authenticate connections to the management console

4. Explore the discovered applications 

5. Run a vulnerability scan and find the most exploitable vulnerabilities

Step 1: Install the ThreatMapper Management Console

To install the management console on a Docker host, first grab the Docker compose file:

wget https://github.com/deepfence/ThreatMapper/raw/master/deployment-scripts/docker-compose.yml

Then spin the console up:

docker-compose -f docker-compose.yml up --detach

It typically takes about 60 seconds for those container images to start up and initialize. Take note of the external IP address of the Docker host, as we will need that for the next step.

For a more scalable, persistent deployment, you can run the management console on a Kubernetes Cluster. Check out the Kubernetes installation instructions.

Step 2: Get the API Key

Access the management console using a web browser (use the IP address noted above) and create the first admin account. From this point onwards, new users in your organization must be invited by an admin user; open registration ceases.

Go to the Settings -> User Management page and take a copy of the API key for your installation:

You’ll need the API key and the IP address of the management console for the next step – registering sensors to call back to the management console.

Step 3: Install ThreatMapper Sensors

ThreatMapper sensors are installed as privileged containers, and support Kubernetes, Docker, and AWS Fargate. ThreatMapper sensors may also be used to monitor and secure bare-metal and virtual machine workloads, where the container is run in a docker runtime in the host.

On Kubernetes, the ThreatMapper sensor is deployed as a DaemonSet, managed using helm:

helm repo add deepfence https://deepfence-helm-charts.s3.amazonaws.com/threatmapper

helm install deepfence-agent deepfence/deepfence-agent \
--set managementConsoleUrl=---CONSOLE-IP--- \
--set deepfenceKey=---DEEPFENCE-API-KEY---

One pod per host is started, and it uses privileged access to standard APIs to monitor network traffic and interrogate running containers.

On Docker, the sensor is installed thus:

docker run -dit --cpus=".2" --name=deepfence-agent --restart on-failure \
--pid=host --net=host --privileged=true \
-v /sys/kernel/debug:/sys/kernel/debug:rw -v /var/log/fenced \
-v /var/run/docker.sock:/var/run/docker.sock -v /:/fenced/mnt/host/:ro \
-e USER_DEFINED_TAGS="" -e MGMT_CONSOLE_URL="---CONSOLE-IP---" \
-e MGMT_CONSOLE_PORT="443" -e DEEPFENCE_KEY="---DEEPFENCE-API-KEY---" \
deepfenceio/deepfence_agent_ce:latest

Initially, the sensors will discover what workloads are running in the platform and identify network flows between each workload.

Step 4: Explore Your Application Topology

Very quickly, you’ll be able to explore your application topology. The ThreatMapper management console builds graph and table-based views that you can use to drill down and explore what workloads are present:

So far, so good, but let’s get to the interesting part – the vulnerability scanning!

The management console needs to bootstrap the vulnerability feeds, which can take anything from 10-45 minutes. Look for a ‘last updated’ message in the Settings -> Diagnosis page for confirmation:

Then select a workload and begin a scan. For workloads in Kubernetes, you can use the ‘Advanced Options’ to scan the entire cluster (nodes and pods). The scan is queued and results are populated in the ‘Vulnerabilities’ report when they become available:

ThreatMapper pulls vulnerability feeds from dozens of different sources. The broadest is the NVD CVE list and ThreatMapper will use that over duplicates in other feeds such as Red Hat or Debian security announcements. ThreatMapper also pulls language-specific feeds for Java, Ruby, NodeJS, Python, and others.

Step 5: Find Your Most Exploitable Vulnerabilities

A list of theoretical vulnerabilities is of little practical use. Vulnerabilities of low severity, requiring local access for exploit are much less concerning than high-severity, network-exploitable vulnerabilities that are present in workloads that receive external network traffic.

The “Most Exploitable Vulnerabilities” report provides exactly that information. In addition to the severity, it considers the attack vector, proximity to the network attack surface, and whether the affected workload has been observed to receive untrusted, external traffic:

You can examine the report in detail, and discover the nature of the vulnerabilities that ThreatMapper has prioritized, and possible remedial actions:

Next Steps

In this tutorial, we showed you how to install the two ThreatMapper components, the management console, and the lightweight sensors. We showed you how to visualize your running services (Topology tab) and your vulnerabilities (Vulnerabilities tab). In your discovered vulnerabilities list, we showed you how to get detailed information about a specific vulnerability, and how to see your most exploitable vulnerabilities, the ones you must fix immediately, based on ThreatMapper’s ability to consider a comprehensive set of runtime conditions to identify how exploitable a vulnerability is within your environment. 

We ZippyOPS, Provide consulting, implementation, and management services on DevOps, DevSecOps, Cloud, Automated Ops, Microservices, Infrastructure, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

Relevant Blogs:

Kubernetes Hardening Tutorial: Pods 

Free DevOps Training and Certifications You Should Know 

How To Connect Stateful Workloads Across Kubernetes Clusters 

4 Kubernetes Anti-Patterns to Avoid