# Lessons learned

Pressure makes diamonds, as the saying goes. I worked on a high-pressure project for a couple of weeks (as in, it needed to be done before we even started), and these are some of the lessons we learned as a team. The lessons are mostly tips and tricks, as we learned a lot on the job.

## General lessons learned

### Way of working

Bring (at least) two developers to the project. One will focus on the algorithm, the other will focus on the code quality and support as much as possible. Notice the choice of words: “focus”. This means that all developers do all the things, but their main task is different.
Don’t underestimate the impact of code quality. Code should be as clear as possible, so that it doesn’t get in the way of solving the business problem. When you’re constantly thinking about what the code does, you’re not thinking about how to solve the business problem. On that note, the first versions were set up as procedural. Refactor to object oriented. OO has advantages over procedural, and it would be a waste to not have access to those advantages. This refactoring was well worth the effort, as we had our codebase audited. No major flaws were encountered during the audit.

### Version control

Get a version control tool in place, and choose the one that is easiest to use. You can share code by emailing .zip files, but that’s too cumbersome. Besides, errors get made. Use git, ask around how to do that, and ignore project managers who tell you not to do this. Even a paid github repository is better than nothing.

### maven

#### Manually include dependencies

It is possible to add dependencies to the build, without the need for those dependencies to be available in a repository. You’ll include them from a /lib folder or something like that:

        <dependency>
<groupId>group.id</groupId>
<artifactId>artifact</artifactId>
<version>1.0</version>
<scope>system</scope>
<systemPath>${project.basedir}/src/test/resources/to-include.jar</systemPath> </dependency>  #### Create complete jar To build the resulting jar with dependencies, use the following command: mvn assembly:assembly -DdescriptorId=jar-with-dependencies #### Version tracking Resource filtering, to update variables in your resources with maven properties. But only variables in certain files, all other files should not be filtered because that might corrupt them:  <build> <resources> <resource> <directory>src/main/resources</directory> <filtering>false</filtering> </resource> <resource> <directory>src/main/resources</directory> <filtering>true</filtering> <includes> <include>project-version.properties</include> </includes> </resource> </resources> </build>  Contents of project-version.properties: version =${build.version}


where ${build.version} is a property in the pom file, along with the format for this timestamp: <properties> <maven.build.timestamp.format>yyyyMMdd-HHmm</maven.build.timestamp.format> <build.version>${maven.build.timestamp}</build.version>
</properties>


To download all sources from the dependencies (when available), type

 mvn dependency:sources

This will allow you to inspect the actual source code when you’re in a debugging session.

#### Skip tests

There are two ways of skipping unit tests:

mvn -DskipTests <task>

Only skips _executing_ the tests. The unit tests will still be compiled

mvn -Dmaven.test.skip=true

Does not compile the tests, and therefore the tests are not executed.

### One piece of software

For testing purposes, we made our program so it ran locally. The same program could run, without modifications, on the server. We used hard-coded paths and keys for the server version, with fallbacks for the local standalone version. This allowed us to focus on the algorithms, and find/fix environments issues quite fast.

### Patching jars

We had to patch the Mendelson jars a few times, before we decided to create a maven build script for the source code.

javac -classpath <jar to be patched>;<jars containing non-local classes used by the class to be compiled> path\to\modified\file.java

Then open the jar with a zip-tool (7zip, for example), and replace the old class with the newly compiled version.

### Logging

Add as much logging as useful. This is probably more than you think. In our case, logging wasn’t showing up. So we wrote a LoggingFacade which wrote its output to the default logging framework, AND to System.out or System.err if needed.

### Debugging

Debugging will provide more information than logging, but is not always possible.
Make one version that run standalone, so you can attach a debugger while developing.
Make sure you can remotely debug the server. Start the server with debug enabled, with the following command-line parameter:

-agentlib:jdwp=transport=dt_socket,address=localhost:4000,server=y,suspend=y

This starts the program in debug mode, listening to debuggers on TCP port 4000. You can choose any port that is convenient for you.

You might need to open an SSH tunnel to your server, listening locally to port 4000, and forwarding it to localhost:4000. Notice that localhost is the localhost of the server, not the localhost from which you make the connection to the server.

Then configure your IDE to connect to a remote application.

### Spring-Boot

One of the avenues we’ve explored was to build a standalone program to intercept and process the messages in a more controllable way. Spring-Boot was introduced for this, but not continued. It is worth exploring these kinds of avenues when you’re stuck, because they might give some insight in how to continue.
Spring-Boot offers quite a lot of extras that we can use for our project, such as a standalone server (run with mvn spring-boot:run). Any application can still be run from within the IDE, because the applications still have a main() function.

About the producing service: https://spring.io/guides/gs/producing-web-service/
About the consuming service: https://spring.io/guides/gs/consuming-web-service/
Switching from application: https://stackoverflow.com/questions/23217002/how-do-i-tell-spring-boot-which-main-class-to-use-for-the-executable-jar

To test the producing service, use Postman (https://www.getpostman.com/apps)
The service can be reached with a POST request on http://localhost:8080
Body type: raw
Body contents can be found on the producing link, file is called “request.xml”

## Project specific

### Decrypting XML

The XML might have been encrypted with a cipher that isn’t available to you. Find the correct cipher in the following section:

	<xenc:EncryptionMethod Algorithm="http://www.w3.org/2009/xmlenc11#rsa-oaep">
<ds:DigestMethod xmlns:ds="http://www.w3.org/2000/09/xmldsig#" Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<xenc11:MGF xmlns:xenc11="http://www.w3.org/2009/xmlenc11#" Algorithm="http://www.w3.org/2009/xmlenc11#mgf1sha256"/>
</xenc:EncryptionMethod>


Take special note of the Digest Method and the Mask Generation Function, as these might not be available to you. You need to use a third party library that implements the exact cipher that is used. In our case that is Apache Santuario.

### Initializing Santuario

Santuario must be initialized before it’s used. However, before initializing the main cryptography engine, the Internationalization framework needs to be initialized. Normally this is initialized with the locale en-US, but only the en (without the _US_ part) properties file is available. This should not be a problem, since this properties file is part of a fallback mechanism. However, in our case, this fallback mechanism doesn’t work.
First initialize Santuario with an empty resource bundle, then initialize the cryptography engine.

### Binary data

In one instance of our project, the binary file had a repeating sequence EF BF BD. This is caused by creating a String from the binary data, and requesting the bytes from that String. Strings and binary aren’t the best of friends, keep them separated!

# JVMCon

On 30 January 2018, the first edition of JVMCon was organised. It was a small conference, only a couple of hundred attendees, but it was sold-out anyway. I attended five sessions, and I will list them in order of least to most awesome.

The first session I attended was by Alexander Yadayadayada (his joke, not mine) about gamification to increase code quality. Most of the ideas came down to “make it a contest!”. And most of the ideas that he tried so far weren’t very succesful. When you make a set of rules for engineers, they will always find ways to game the system and get ahead without any effort. For example, one point per unittest: create empty unittests. Give someone a taco on slack for doing something awesome: “Hey, I don’t have any tacos, could anyone throw me some?” But the biggest problem seems to be that people usually decide to stop playing, because the game isn’t balanced properly.

Hadi Hariri’s session was named Kotlin 102. It didn’t cover the basics of Kotlin, but a bit more advanced stuff. Hadi had a live-demo presentation, which is always impressive. However, he was talking to a tough crowd. Perhaps the reason was because his session was after lunch. Or maybe Kotlin isn’t known well enough yet to get into the more advanced stuff.

The third session in this list is that of Angelo van der Sijpt: What you didn’t know you wanted to know about the JVM. This one started to tickle my nerdy-senses. He spoke of Java, bytecode, C and even assembler, right down to the individual instructions that will be executed by the CPU. Then there were bits about how the memory is really used. There was a quizz: is this word a CPU instruction or not? Hint, IDCLIP is not. It was an interesting talk, but a bit too advanced for me.

Then there is Venkat Subramaniam’s talk about Java 9 Modularization. Modules are here, and they are here to stay. However, I don’t think the world is ready for it just yet. And with the new Java release cycle (And totally messed-up versioning system. WTF Oracle, really?), I don’t think there will be many production systems that will run Java 9. Anyway, Venkat started his talk with the remark “If there are any questions, please interrupt”. Then he started to spew information faster than the audience could process it. So, when you wanted to ask a question, he was already three topics ahead. He also had a live demo, which didn’t always go as planned. But then again, he disguised a typo with a joke: “If you do this, things go so wrong that you don’t even get an error. You get an ERRRO!” If you want to go to an information rollercoaster, see Venkat live.

Take the sum of the awesomeness of all the previous talks, and then multiply it with the sum of their nerdyness, and you’re still not even close to the last talk. This one was in a league of it’s own: Don’t hack the platform? by Jan Ouwens. If you need inspiration for messing with your colleagues, this is the one for you. From Unicode-hacks to overwriting JVM constants to changing complete method implementations. …On a running system. …remotely. This guy had some evil, evil hacks.