Not Your Grandfather’s Empire I’ve wanted to put this blog together since returning home from DEFCON. Anytime we ran into someone who recognized our swag, they mentioned how much they loved Empire back in the day and didn’t realize it was being actively maintained. This made me reflect on all [...]
Jacob Krasnov | Anthony Rose
This blog is going to be the first entry in a series that goes over some of the fundamental concepts of Offensive Security. There are a lot of blog posts out there about all of these topics, but we wanted to build a series with some continuity to it and is friendly to beginners and veterans alike. While many of these posts will be “basic”, we find that even veterans often get many of these fundamental concepts muddled. We also wanted to bring in some more formal test engineering concepts as both Cx01N and my background are in engineering and operational testing.
What is Offensive Security?
When we started planning this series, this was a question that was brought up and while we “knew” what it was, we couldn’t come up with a good definition. It was more of a “you know it when you see it” kind of thing and that’s not a very good answer at all. So we started doing some research and thinking and came up with two definitions. The first is an adaption of the definition for red teaming from the Red Team Journal [1]:
The practice of testing security measures from an adversary or competitor’s perspective.
In other words, this definition says that Offensive Security is about testing security postures from the viewpoint of an adversary or competitor. For example, let’s pretend that a product has the best security ever constructed for protecting user passwords. However, a competitor wants to reverse engineer the source code of the product to make a copy. If the product doesn’t have protections in place to prevent a competitor from stealing their product then the overall security of the product is inadequate, regardless of how well it performs its intended purpose.
Organizations can improve their security through the understanding and application of adversarial tactics. Applying test and system engineering theory leads us to our second definition:
Validation of security controls or postures through negative testing.
For people that are unfamiliar with formal testing concepts, there are two key parts to this definition. The first is the “Validation of security controls or postures.” Validation should not be confused with Verification as the two are very different phases of testing.
Verification is testing to see that something was built to specification or requirements. For example, a system may have a requirement to encrypt stored passwords. As long as the passwords are encrypted in some way, it would pass verification testing.
Validation is testing a system to see if it meets the intent of what the requirements were attempting to achieve. Continuing with the password encryption example, if a system were found to use DES encryption to store passwords, that would likely cause some concern. The intent of requiring the passwords would be looked at and in this case, the requirement is meant to prevent an attacker from using passwords if the database was compromised. DES encryption would clearly not achieve that goal and the system would fail validation testing [3].
Going back to our Offensive Security definition, red teams and pentesters are attempting to validate that an organization’s systems and processes are achieving the intended goals.
The second part of our technical description is “through negative testing.” This brings in another set of definitions from test engineering: positive testing and negative testing. These are often misunderstood and confusing to many, but the concepts are quite simple. Positive testing is traditionally defined as testing an application or system to verify that an anticipated input produces the expected behavior. On the other hand, negative testing is testing an application or system with unanticipated values to confirm that the system responds in a controlled manner.
For this series, we are going to use a modified version of these definitions:
Positive Testing: Testing an application or system for anticipated responses.
Negative Testing: Testing an application for unexpected responses.
With these definitions, we can now understand the definition presented above. Offensive Security can be thought of as the validation that security controls or postures are performing their intended purposes through the use of inputs that attempt to illicit unintended behavior.
Red Teaming vs. Penetration Testing
While Red Teaming and Penetration Testing are not the only types of testing in Offensive Security, they are the primary tests run by companies like BC Security and are what this series will focus on. They are also where we see the most confusion when dealing with customers who often use the terms interchangeably and misunderstand their testing objective. Red Teaming and Penetration Testing objectives are fundamentally different, which makes the two types of tests distinct. Red Team assessments typically aim to test processes, while penetration tests focus more on assessing the effectiveness of the technical implementation. Let’s take a few minutes to do a more in-depth examination of Red Teaming and Penetration Testing.
The simple definition is that it is the utilization of friendly forces as adversarial forces for the purpose of training and testing defenses. Its roots come from the use of Opposition Forces (OpFor) in military exercises to familiarize and prepare troops for combat. Probably the most well known of these is the Air Force’s Red Flag exercises conducted at Nellis Air Force base several times a year.
During the Vietnam war, USAF pilot losses were extremely high, at one point reaching a kill ratio of less than 1 (they lost more than one plane for every plane they shot down). These losses were clearly unacceptable, and studies showed that pilots were the most vulnerable during the first 10 combat missions. To remedy this, the Air Force instituted Red Flag. One of the essential parts of this exercise is the implementation of Red Air, where specially trained pilots flew against the Air Force pilots employing the doctrine used by the enemy pilots they would operate against. As a result, survival rates drastically increased [5].
Red Flag gives us a great example of what Red Teaming is looking to achieve. It trains operators on the tactics employed by enemies and allows for the refinement of Blue Force tactics, techniques, and procedures (TTPs). It is a test and training of the processes used and employed, not of the technical capabilities or defenses employed. For example, it is not testing whether the flares that Air Force assets use are effective. It tests at what point in the engagement, those flares should be used and if pilots know when to use them. Similarly, a Red Team assessment’s main goal should not be to identify if the password encryption used is the best one out there, but whether the phishing training is working, the blue team has proper response protocols, if they know how to properly use their monitoring tools, or if those tools are properly configured.
On the other hand, Penetration Testing is attempting to test the technical capabilities and implementation of an organization. Going back to our Air Force analogy, this is the type of testing that looks at how effective the flairs are at drawing enemy weapons or if the plane’s maneuverability is suitable enough to compete with adversaries.
The inverse pyramid, which can be found in Joe Vest’s blog and book Red Team Development and Operations, displays the differences in breadth and depth for each type of test. A Penetration Test seeks to determine if the whitelisting application implementation is effective or the anti-phishing software works as intended. It also should attempt to find as many issues as possible. The Red Team assessment has a narrow scope, which simply tests the organization against a specified threat. The team emulates that threat but doesn’t attempt to find all possible issues. A Penetration Test should attempt to cast a wider net and enumerate as many issues as possible. Rapid7 has a pretty helpful article if you are interested in reading more about the differences between Red Teaming and Penetration Testing.
Where do we go from here?
Now that we have a good understanding of what Offensive Security is and the types of testing typically employed, we can move on to the next phase. In the next post of this series, we will discuss the phases involved in planning a test. The phases of a Penetration Test and Red Team assessment are pretty similar, but the planning can be quite different. After we cover the phases, we will then move into more technical posts on the tools and tactics that can be used in each phase.
[1] https://redteamjournal.com/red-teaming-and-alternative-analysis/
[3] https://en.wikipedia.org/wiki/Verification_and_validation
[4] https://developer.salesforce.com/blogs/2019/09/codelive-positive-and-negative-unit-testing.html
There are a lot of significant changes to Empire, so we thought it would be a good idea to walk through them in a little more detail.
