CVE-2016-1238 and the hazards of testing frameworks

Because I have lost confidence in the approach taken by those in charge of fixing this problem, I have decided to do a full disclosure series on CVE-2016-1238.  As mentioned in a previous post, the proposed fixes for the minor versions don't even remotely fix the problem  and at most can provide a false sense of security.  For this reason it is extremely important that sysadmins understand how these exploits work and how to secure their systems.

A lot of the information here is not specific to this CVE but concerns security regarding running tests generally.  For this reason while this CVE is used as an example, the same basic concepts apply to PostgreSQL db testing, and much more.

As I mentioned in a previous post, prove cannot currently be run safely in any directory that is world writeable, and the current approach of making this a module problem make this far worse, not better.  Moreover this is not something which can be patched in prove without breaking the basic guarantee that it tests the application as it would run on the current system's Perl interpreter and if you break that, all bets are off.

All the exploits I cover in this series are exploitable on fully patched systems.  However they can be prevented by good system administration.  In every case, we will look at how system administration best practices can prevent the problem.

One key problem with the current approach is that it fails to differentiate between unknowing inclusion and user errors brought on simply by not understanding the issues.  The latter has been totally disregarded by the people attempting stop-gap patches.

Test harnesses generally are insecure by design.  The whole point is to execute arbitrary code and see what happens and with this comes a series of inherent risks.  We accept those risks because they are important to the guarantees we usually want to make that our software will perform in the real world as expected.  Moreover even the security risks inherent in test harnesses are good because it is better to find problems in an environment that is somewhat sandboxed than it is in your production environment.

So testing frameworks should be considered to be code injection frameworks and they can be vectors by which code can be injected into a tested application with system security implications.

Understood this way, testing frameworks are not only insecure by design but this is desirable since you don't want a testing framework to hide a problem from you that could bite you in production.

This is a generic problem of course.  A database testing framework would (and should) allow sql injection that could happen in the tested code so that these can be tested against.  So whether you working with Perl, Python, PostgreSQL, or anything else, the testing framework itself has to be properly secured.  And in each platform this means specific things.

In PostgreSQL, this means you need to pay close attention to certain things, such as the fact that the database tests should probably not run as a superuser for example since a superuser can do things like create functions wit system access.

In Perl, one of the most important things to consider is the inclusion of modules from the current working directory and the possibility that someone could do bad things there.

What Prove Guarantees

Prove guarantees that your perl code will run, in its current configuration, in accordance with your test cases.  This necessarily requires arbitrary code execution and arbitrary dependency requirements resolved in the way Perl would resolve them on your system.

Prove guarantees that the guarantees you specify in your test cases are met by your current Perl configuration.  It therefore cannot safely do any extra sandboxing for you.

How Prove Works

The basic architecture of prove is that it wraps a test harness which runs a specified program (via the shebang line) parses its output assuming it to be in the test-anything protocol, and generates a report from the rest.  For example if you create a file test.t:

#!/bin/bash

echo 'ok 1';
echo 'ok 2';
echo '1..2';

and run prove test.t

You will get a report like the following:

$ prove test.t 
test.t .. ok   
All tests successful.

What prove has done is invoke /bin/bash, run the file on it, parse the output, check that 2 tests were run, and that both printed ok (it is a little more complex than this but....), and let you know it worked.

Of course, usually we run perl, not bash.

An Attack Scenario

The most obvious attach scenarios would occur with automated test environment that are poorly secured.  In this case, if prove runs from a directory containing material more than one user might submit, user A may be able to inject code into user B's test runs.

Suppose user A has a hook for customization as follows:

eval { require 'custom.pl' };

Now this is intended to run a custom.pl file, at most once, when the code is run, and it checks the current @INC paths.  If this doesn't exist it falls back to the current working directory, i.e. the directory the shell was working in when prove was run.  If this directory shares information between users, user b can write a custom.pl file into that directory which will run when user A's scheduled tests are run.

The code would then run with the permissions of the test run and any misbehavior would tie back to user A's test run (it is harder to tie the behavior to user B).  In the event where user A's test run operates with more system permissions than user B's, the situation is quite a bit worse.  Or maybe user B doesn't even have tests run anymore for some reason.

Now, it has been proposed that prove prune its own @INC but that doesn't address this attack because prove has to run perl separately.  Additionally separation of concerns dictates that this is not prove's problem.

Behavior Considered Reasonably Safe

As we have seen, there are inherent risks to test frameworks and these have to be properly secured against the very real fact that one is often running untrusted code on them.  However, there are a few things that really should be considered safe.  These include:

• Running tests during installation of trusted software as root.  If you don't trust the software, you should not be installing it.
• Running tests from directories which store only information from a single user (subdirectories of a user's home directory for example).

Recommendations for Test Systems

Several basic rules for test systems apply:

1. Understand that test systems run arbitrary code and avoid running test cases for automated build and test systems as privileged users.
2. Properly secure all working directories to prevent users from unknowingly sharing data or test logic
3. Running as root is only permitted as part of a process installing trusted software.