detecting nasty class/jar files?

[Andreas Leitgeb]

Before I run foreign java-programs on my machine, I generally
like to know what they *can* do. Assuming there are no
dll's involved (otherwise I wouldn't idly run the program,
anyway)

So I scan the constant pools of all classes(*). I'd expect that
any platform method call would have to appear there, so if nothing
suspicious (especially no references to ClassLoader.defineClass
and reflection-stuff) is found in the constant pools, are there
other tricks left that I'd need to check for possibly dangerous
API calls?

Are there any harmless looking(**) methods that would make it
possible to get new classes defined, or methods called through
dynamic strings?

I'm aware of SecurityManagers, but I don't trust my skills
to set one up for my demands without leaving too much open.
This is about "proglets" I download from web for offline
execution, with not exactly the same set of restrictions
as applets (some more some less). Any such proglet that
tries to call inappropriate stuff or open door to dynamic
code execution I'd throw away rather then let run.

(*): As written out by javap with appropriate options (-c,-verbose)
(**): to a pair of scjp-level eyes ;-)

 

[Chris Uppal]

So I scan the constant pools of all classes(*). I'd expect that
any platform method call would have to appear there, so if nothing
suspicious (especially no references to ClassLoader.defineClass
and reflection-stuff) is found in the constant pools, are there
other tricks left that I'd need to check for possibly dangerous
API calls?

I'd be a lot more worried about uncontrolled use of FileOutputStream, and its
friends. And of FileInputStream. And of network connections. And probably
more stuff too. Not to mention all the standard API's which use those
features internally and which could be suborned (by accident or in malice) to
do something you don't want (e.g. /any/ API which takes a filename as a String
is potential a point of abuse).

Are there any harmless looking(**) methods that would make it
possible to get new classes defined, or methods called through
dynamic strings?

As far as I know, the only routes though which you can define an arbitrary
class dynamically are JNI or a classloader. But note that if you can create an
ordinary file on the classpath, then you may be able to load it without using
any /explicitiy/ reflective code.

I'm aware of SecurityManagers, but I don't trust my skills
to set one up for my demands without leaving too much open.

Then I suggest that you /develop/ those skills. Right now, it doesn't sound to
me as if you are achieving anything like the level of security (by eyeballing
constant pools) as you would get from even an incompletely nailed-down security
manager.

Another possibility would be to move the whole question outside the Java space,
and run your proglets as a different user with severely restricted access to
the rest of the system. How easy that would be depends on how tightly
configured your system is already (maybe trivial, maybe almost impossible).

-- chris

 

[Andrew Thompson]

Before I run foreign java-programs on my machine, I generally
like to know what they *can* do.

Not an answer to your question directly, but if you
load the project
a) using webstart (in a 'no permissions' environment), or
b) from an untrusted applet
...it will be limited to a small set of 'safe' actions.

If it fails, you should get stack traces describing
what it was attempting.

Andrew T.

 

[Andreas Leitgeb]

Not an answer to your question directly, but if you
load the project
a) using webstart (in a 'no permissions' environment), or

It indeed isn't what I was after. My goal is rather a
static analysis of a program, given only it's class files,
to see, what methods it *can* possibly invoke. (and some
hints for platform methods that could open doors in a
less obvious way than reflection and defineClass)

It's obvious, that if the program calls Runtime.exec(...)
or accesses files, then it is obviously "possibly not harmless".

It's only a tiny bit less obvious, that by defining new classes
from strings, it can hide attempts at the first point and that
through reflection, methods can be called that are not mentioned
in the constant pool.

I'm asking for even less obvious other "holes" to subvert
this kind of static analysis, e.g. whether some swing class
could be misused to call methods named only by Strings.
How about (de-)serialization? (I don't see a way, but there
might be something I miss) or RMI, Corba? (I'd probably
distrust these in doubt)

PS: sorry, I should have mentioned "static analysis" right in my
first post, but this term occurred to me only afterwards.

 

[Oliver Wong]

My goal is rather a
static analysis of a program, given only it's class files,
to see, what methods it *can* possibly invoke. (and some
hints for platform methods that could open doors in a
less obvious way than reflection and defineClass)

If you allow for your analysis-engine to also have access to the
source code of Sun's SDK library, you might be able to do a control-flow
analysis to see if it's ever possible for SecurityException to get thrown.

Note that you'll need Sun's source code, and not merely their
precompiled library, as sometimes the javadocs declare a method (e.g.
java.io.File.canExecute()) as throwing SecurityException, but it's thrown
by native code, and thus probably cannot be seen by just analyzing the
bytecode -- you'll have to actually parse the JavaDoc comments.

- Oliver

 

[a24900]

On Mar 13, 10:45 am, Andreas Leitgeb

It indeed isn't what I was after. My goal is rather a
static analysis of a program,

Waste of time. There is no single point defining which classes belong
to an application. An application can be split among multiple jars,
multiple directories, multiple URLs and whatever a classloader can dig
up. Your static analysis can easily miss a source.

Let the VM do its job. Tighten your policy file and run the VM with an
activated security manger.

 

[Oliver Wong]

On Mar 13, 10:45 am, Andreas Leitgeb

Waste of time. There is no single point defining which classes belong
to an application. An application can be split among multiple jars,
multiple directories, multiple URLs and whatever a classloader can dig
up. Your static analysis can easily miss a source.

Let the VM do its job. Tighten your policy file and run the VM with an
activated security manger.

A static analyzer and a VM are different tools and tend to be used in
different situations. VMs are typically useful if you actually want to run
the program. Static analyzers are typically useful if you want to obtain
information about a program without actually running it. Usually you can't
interchange one for the other, unless the program is completely
deterministic (e.g. its behaviour doesn't depend on user input, or the
contents of files, etc.)

- Oliver

 

[Andreas Leitgeb]

My goal is rather a static analysis of a program,...
to see, what methods it *can* possibly invoke.

[checking for platform-methods that can throw SecurityException]
Note that you'll need Sun's source code, and not merely their
precompiled library, as sometimes the javadocs declare a method

So, actually I could just as well check the html javadocs
for the SDK

I'd lookup all platform methods called from the program's code,
in the javadoc, and see if any of these is documented to throw an SE.

Thanks. That might identify me some further suspicious methods...

 

[Andreas Leitgeb]

On Mar 13, 10:45 am, Andreas Leitgeb

Waste of time. There is no single point defining which classes belong
to an application. An application can be split among multiple jars,
multiple directories, multiple URLs and whatever a classloader can dig
up. Your static analysis can easily miss a source.

I don't quite seem to understand.
If I run a jar-file with "java -jar app.jar", the only
"other" class files that can be picked up are those
explictly mentioned in the classpath plus those in
in the jre's lib/ext/ directory.

If I have some toy-program.jar, I don't add any extra jar-files
to the classpath, nor do I put jar files into the ext dir, unless
they contain only packages that I would recognize as belonging to
these jar-files (if the toy-program was to try to call them).

Let the VM do its job. Tighten your policy file and run the VM with an
activated security manger.

An application can catch all the SecurityExceptions it runs against,
so I don't ever see (not even if I single-step through it in a
debugger), what the app really does, until I once (in a weak moment)
run it unrestricted.

 

[Joshua Cranmer]

I don't quite seem to understand.
If I run a jar-file with "java -jar app.jar", the only
"other" class files that can be picked up are those
explictly mentioned in the classpath plus those in
in the jre's lib/ext/ directory.

Not necessarily. I have seen code (a signed applet) that downloaded
itself over the internet through means of external jars and its own
ClassLoader. URLClassLoader, for example, is capable of loading classes
from some location on the internet.

If I have some toy-program.jar, I don't add any extra jar-files
to the classpath, nor do I put jar files into the ext dir, unless
they contain only packages that I would recognize as belonging to
these jar-files (if the toy-program was to try to call them).


An application can catch all the SecurityExceptions it runs against,
so I don't ever see (not even if I single-step through it in a
debugger), what the app really does, until I once (in a weak moment)
run it unrestricted.

Then write your own security manager. It's not too difficult.

 

[Andreas Leitgeb]

... I have seen code (a signed applet) that downloaded
itself over the internet through means of external jars and its own
ClassLoader. URLClassLoader, for example, is capable of loading classes
from some location on the internet.

But to do this the primary jar file would already have to include a call
to some networking class (to download the extra jar), and some reference
to defineClass() (to be called from the custom classLoader)

I would see, *that* some extra classes might be defined dynamically,
and that would be enough information to classify the program principially
as "possibly harmful".

 

[pkriens]

Before I run foreign java-programs on my machine, I generally
like to know what they *can* do. Assuming there are no
dll's involved (otherwise I wouldn't idly run the program,
anyway)

With a security manager your code can do anything and it is easy to
obfuscate what you do. The Class.forName allows you access to any
class on the system and reflective access allows you access to do
anything you like. Again, there is NO security in Java without a
SecurityManager.

Kind regards,

Peter Kriens

 

[Andreas Leitgeb]

With a security manager your code can do anything and it is easy to
obfuscate what you do.

I don't want to obfuscate anything. Quite the opposite: I want
to see if anyone else (whose jar-file I'm about to analyse) tried
to obfuscate something in it.

The Class.forName allows you access to any class on the system

Yes, sure it does, but it must still be called from the jar-file
in the first place.
A jar-file that calls Class.forName() and reflection is
then suspicious. (as far as I think, Class.forName alone
is not yet dangerous, but I could be wrong there, of course.)

Fortunately, the only way for java-programs to dynamically
add new program code is through a small number of platform
methods. If none of those are called, anything else is
already *all* that's possibly called. (And if any of those
get called, then the program is no longer trustable from
statical point of view - I might of course still run it
with an appropriate securitymanager.)

and reflective access allows you access to do anything you like.

It does, but it can't hide the fact when it does.

 

[Oliver Wong]

Yes, sure it does, but it must still be called from the jar-file
in the first place.
A jar-file that calls Class.forName() and reflection is
then suspicious. (as far as I think, Class.forName alone
is not yet dangerous, but I could be wrong there, of course.)

Classes can have static initializer, thus triggering for code to run
as soon as they are loaded, even if none of the methods (static or
otherwise) or constructors on that class are called.

- Oliver

 

[Andreas Leitgeb]

Classes can have static initializer, thus triggering for code to run
as soon as they are loaded, even if none of the methods (static or
otherwise) or constructors on that class are called.

So, anyway it needs to call .forName() and have that reference in it.
Then, if the class, that gets loaded is in the jar-file, then it's
also under statical analysis.
If it is a foreign one, then it must be one in the classpath, and
I trust, that no class in the JRE, nor in my ext directory will
do anything nasty in it's static initializer block.

But it's a good point, anyway. Better put .forName()
onto my distrust-list, as well, even if no reflection
appears to be used.

Thanks for pointing that one out.

 

[Oliver Wong]

So, anyway it needs to call .forName() and have that reference in it.
Then, if the class, that gets loaded is in the jar-file, then it's
also under statical analysis.
If it is a foreign one, then it must be one in the classpath, and
I trust, that no class in the JRE, nor in my ext directory will
do anything nasty in it's static initializer block.

.forName() can also specify a class loader, which might be a custom
one which loads classes from a location other than the classpath, the JRE
and your ext directory. It might, for example, dynamically create the
bytecode for the new class, from a byte array.

- Oliver

 

[Andreas Leitgeb]

.forName() can also specify a class loader, which might be a custom
one ...

but then, again, some custom classloader must exist in the jar-file
and that one would have to call defineClass(), the #1 red alert.

 

[Tom Hawtin]

I'm asking for even less obvious other "holes" to subvert
this kind of static analysis, e.g. whether some swing class
could be misused to call methods named only by Strings.
How about (de-)serialization? (I don't see a way, but there
might be something I miss) or RMI, Corba? (I'd probably
distrust these in doubt)

There are so many things that can be done by an adversary. You need to
get lucky right across all APIs. An attacker only needs one hole. Your
chances of anything other than security by obscurity are slim.

You can't realistically anticipate what someone might do or where there
might be holes. Attacks can use surprising mechanisms. You mention Swing
and serialisation. Even with the native Java security mechanism holes
have been found (by really smart people

http://sunsolve.sun.com/search/document.do?assetkey=1-26-102622-1
http://sunsolve.sun.com/search/document.do?assetkey=1-26-102732-1
http://sunsolve.sun.com/search/document.do?assetkey=1-26-102731-1

So, my suggestion is that you learn about Java security. Static analysis
is a truly wonderful thing, but I don't think it's an appropriate choice
here.

Tom Hawtin

 

[Thomas Kellerer]

but then, again, some custom classloader must exist in the jar-file
and that one would have to call defineClass(), the #1 red alert.

No, the class could use URLClassLoader which is part of the JDK. You
should then also scan for usages of that, not only "custom" classloaders

Thomas

 

[Chris Uppal]

There are so many things that can be done by an adversary. You need to
get lucky right across all APIs. An attacker only needs one hole. Your
chances of anything other than security by obscurity are slim.

It's something of a truism of security that, in any complex system, attempting
to create a "blacklist" of disallowed operations is going to leave holes. The
"normal" (i.e. best) practise is to use a whitelist of allowed operations, and
forbid everything else.


But I wonder how close you could get...

I think (ignoring the possibility of bugs in the JVM) you could automate part
of the generation of the whitelist. A method is safe if and only if:

1a) it cannot be overridden, or
1b) it is only overridden by safe methods;
2) it is not native[*];
3) it does not use the facilities of sun.java.unsafe.* or any other
JVM-level backdoor[**];
4) it calls no other method which is not itself safe;

[*] There are several independent reasons to treat any code which touches JNI
as unsafe:
a) JNI code is completely unregulated and can do /anything/ (e..g
substitute its own code for any chosen class's own native methods...
b) JNI code cannot be verified by the above algorithm, so you have to
assume that it's unsafe.
c) JNI code is likely to include exploitable holes (like buffer overflows,
etc).
d) (most importantly) ultimately, any code which /does/ anything/ is going
to do it via JNI so this is the place to look for things which shouldn't
be allowed, or which are susceptible to abuse.

[**] There are quite a few JVM-level backdoors, but I think most of them could
be certified safe by inspection.


Note that:

+ (1b) can only be verified by whole-program analysis unless (1a) is
already satisfied.
+ Whole-program analysis is impossible if there is any mention of
classloaders anywhere,
(even in platform code, and even if restricted to platform supplied
Classloader subclasses).
+ (4) excludes most reflective calls.
+ (4) excludes instantiating or referring to any [member of any] class
whose <clinit> method is not safe.


At a rough guess, and assuming you whitelist System.out.println() (and
friends), you might be left with enough features to be able to certify as safe
solutions to the exercises from the first couple of chapters in a beginner's
Java book ;-)


But then, perfect security is a mirage -- there is no such thing, and even
thinking in those terms is misleading. It would be better to consider if any
of the above techniques would usefully /improve/ the safety of the application
(bearing in mind the degree of risk involved and the costs of reducing it).

-- chris