Nyctalopia - minimalistic flow obfuscator

How to protect intellectual property against script kiddies and other inferior forms of life?

Motivation

I do believe in open source and I am committed to the development of free software. I do not prefer tiny-soft software enterprises who compensate their inferior quality technical solutions with obscurity (cultural excursion). The fact that only the developer of a solution knows the internals does not impose real qualities; a great solution stays just as valuable if the implementation details are leaked/disclosed. Once can read nice articles on this topic.

Knowledge is power, he who has outstanding ideas and patent, who creates smart solutions, deserves to lead a high quality of life. In case he decides to share his intellectual property and assets with the public, he may provide limitations as of who can use his work for what purposes under which conditions. Thus we have patents and licenses.

Unfortunately, some do not respect the efforts and property of others, copy work of others and market it as their own creation, although it is protected by copyright, e.g. licensed under an open source license. Obviously, one could go to court, but doing so might not be worth it. Such a frustrating situation was the driver for this one night hack (1 day project).

Technical background

Java source is compiled to jvm bytecode which in its structure is similar to machine but but it consists of simpler constructs as it is not heavily optimized. The jvm reads this intermediate representation and compiles it to machine code instructions for the underlying hardware on the fly. The main advantage of this concept is portability of byte-code between platforms, an often mentioned drawback is degraded speed of execution. Obviously, the java compiler only performs relatively simple transformation compared to the heavy optimizations a native compiler typically does. As a side effect, it is much easier to reverse engineer java bytecode as opposed to decompiling native code to C. There are many effective java decompilers out there, these mostly graphical tools do not require deep expertise.

There are tools which transform bytecode by stripping some parts - typically debug information and dead code - and shorten stored names of internal variables, resulting in smaller, somewhat faster binaries. As a consequence, reverse engineering becomes harder, the decompiled source is harder to read (understand). These tools are called obfuscators or bytecode optimizers. Their use does not provide much protections against reverse engineering. There are more advance obfuscators, which also scramble the control flow, which means they also alter the execution of the program to some extent in a way that the program still 'does the same thing'; these are called control flow obfuscators.

With basic theoretical knowledge it can be deduced that arbitrary bytecode sequenced cannot be transformed to java, just like the fact that for any given bytecode sequence, infinite alternative bytecode sequenced can be provided which "do the same thing".

Objective

To transform java bytecode in a way that the behaviour is not affected, but decompilation results in either an error or an output that is not valid java code, and cannot easily be transform to it. An important aspect is to issue the least changes that yield the most results. As the guinea pig I used the jd-gui decompiler which, in my opinion, is not most effective one at the time of this writing.

It is important to understand, that with the necessary knowledge and effort, theoretically, any program can be analyzed, understood and reconstructed. Security through obscurity might scare off the inferior ones, but will not stop well prepared attackers, merely slow them down. Currently, the goal is to defend against parasites.

How?

First one has to be somewhat familiar with compiler technology and jvm bytecode. As we aim to introduce minimal changes to the bytecode, I was seeking for instruction sequences that could be injected into certain locations without the need to perform complex (control flow) analysis on the methods: an idempotent, non intrusive solution, that will confuse the decompiler to the needed extent.

As the first idea I thought I could insert some instructions right after the return instructions of the method. As was dead code, I would surely not influence the execution of the program. Of course there might be instructions present after RETURN which are not dead code by any means - code execution could well jump to these location, just take any method with multiple RETURNs. One has to watch out for jumps and offsets, or use a framework that takes of this so we do not have to. I hand-coded a proof of concept solution for this with the help of jasmin, which is a java assembler.
It seems that POP ATHROW injected after /.?RETURN/ instructions result into just enough confusion in case of methods of at least medium complexity. To automate the transformation I have throws together a small tool that takes and convert a jar into another one containing already bit bashed java bytecode. The tool uses the asm library.
During testing I have found it might be worth to insert some garbage before /.?RETURN/ instructions as well to spice up the situation even more. My idea was to create a dead branch (ICONST_0 IFEQ), over-fill and over-pop the stack or perform type-mismatched push-pop sequences that do not have any java counterpart. During my experiments I found that the bytecode verifier (that checks each class before loading) does a great job at spotting such atrocities - although the hostile instructions are dead code meaning they could not ever cause runtime problems, the verifier refused to load those fudged, infected classes.
Valid but hackish flow control structures yielded satisfactory results, having minimized the code a never executing infinite loop became the final version I stayed with. Kind of a surprise but this is just enough to fool the decompiler.

			
/*
 * Nyctalopia - minimalistic flow obfuscator
 * 
 * Copyright 2011 Tibor Bősze <tibor.boesze@gmail.com>
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package hu.lithium.nyctalopia;

import org.objectweb.asm.ClassAdapter;
import org.objectweb.asm.ClassVisitor;
import org.objectweb.asm.Label;
import org.objectweb.asm.MethodAdapter;
import org.objectweb.asm.MethodVisitor;
import org.objectweb.asm.Opcodes;

class ObfuscatingClassAdapter extends ClassAdapter {

	ObfuscatingClassAdapter(ClassVisitor cv) {
		super(cv);
	}

	@Override
	public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
		MethodVisitor mv = cv.visitMethod(access, name, desc, signature, exceptions);
		if (mv != null) {
			mv = new ObfuscatingAdapter(mv);
		}
		return mv;
	}

	static class ObfuscatingAdapter extends MethodAdapter implements Opcodes {

		ObfuscatingAdapter(MethodVisitor mv) {
			super(mv);
		}

		@Override
		public void visitInsn(int opcode) {
			if (RETURN >= opcode && opcode >= IRETURN) {
				Label l = new Label();
				Label k = new Label();
				mv.visitLabel(k);
				mv.visitInsn(ICONST_0);
				mv.visitJumpInsn(IFEQ, l);
				mv.visitJumpInsn(GOTO, k);
				mv.visitLabel(l);
				mv.visitInsn(opcode);
				mv.visitInsn(POP);
				mv.visitInsn(ATHROW);
			} else {
				mv.visitInsn(opcode);
			}
		}

	}

}

Kind of typical and funny that finding out and implementing the bytecode transformation itself, chosing the appropriate pattern to be injected and testing it took less time and less lines of code than reading a jar, doing housekeeping and other side-activities. During creation of the resulting jar file, besides transforming *.class files some small optimizations take place like omitting directory entries, on the fly compression and enhancing the manifest file with a tagging attribute.

			

/*
 * Nyctalopia - minimalistic flow obfuscator
 * 
 * Copyright 2011 Tibor Bősze <tibor.boesze@gmail.com>
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *    http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
package hu.lithium.nyctalopia;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.jar.Attributes;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.jar.JarInputStream;
import java.util.jar.JarOutputStream;
import java.util.jar.Manifest;

import org.objectweb.asm.ClassReader;
import org.objectweb.asm.ClassWriter;

public class JarProcessor {

	public final static String copyright = "Nyctalopia - minimalistic flow obfuscator\n"
		+ "Copyright 2011 Tibor Bősze <tibor.boesze@gmail.com>\n"
		+ "Licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0)\n";
	
	private static byte[] processClass(InputStream is) throws IOException {
		ClassWriter w = new ClassWriter(ClassWriter.COMPUTE_MAXS);
		new ClassReader(is).accept(new ObfuscatingClassAdapter(w), 0);
		return w.toByteArray();
	}

	private static JarOutputStream createJarOutStream(JarInputStream input, String path) throws IOException {
		Manifest m = input.getManifest() == null ? new Manifest() : input.getManifest();
		Attributes attrs = m.getMainAttributes();
		if (!attrs.containsKey(Attributes.Name.MANIFEST_VERSION.toString())) {
			attrs.putValue(Attributes.Name.MANIFEST_VERSION.toString(), "1.0");
		}
		attrs.putValue("Created-By", "1.0 (Nyctalopia)");
		
		JarOutputStream output = new JarOutputStream(new FileOutputStream(path), m);
		output.setLevel(9);

		return output;
	}
	
	private static void transformJarContent(JarInputStream in, JarOutputStream out) throws IOException {
		byte[] buf = new byte[4096];
		
		JarEntry entry = null;
		while((entry = in.getNextJarEntry()) != null) {
			if (entry.isDirectory() || JarFile.MANIFEST_NAME.equalsIgnoreCase(entry.getName())) {
				System.out.println("skipping " + entry.getName());
				continue;
			}
			
			JarEntry newEntry = new JarEntry(entry);
			newEntry.setMethod(JarEntry.DEFLATED);
			newEntry.setCompressedSize(-1);
			
			if (entry.getName().endsWith(".class")) {
				System.out.println("weaving " + entry.getName());
				byte[] byteCode = processClass(in);
				out.putNextEntry(newEntry);
				out.write(byteCode);
				out.closeEntry();
			} else {//copy
				System.out.println("copying " + entry.getName());
				out.putNextEntry(newEntry);
				int c = 0;
				while ((c = in.read(buf)) != -1) {
					out.write(buf, 0, c);
				}
				out.closeEntry();
			}
		}
	}
	
	
	public static void main(String[] args) throws IOException {
		System.out.println(copyright);
		
		if (args.length != 2) {
			System.out.println("Usage: <input_jar> <output_jar>");
			return;
		}

		JarInputStream in = null;
		JarOutputStream out = null;

		try {
			in = new JarInputStream(new FileInputStream(args[0]));
			out = createJarOutStream(in, args[1]);
			
			transformJarContent(in, out);
			
		} finally {
			if (in != null) {
				in.close();
			}
			if (out != null) {
				out.close();
			}
		}
	}

}