Troubleshooting Power-Line Communication

Power-Line Communication, turning the power-grid in your home into a computer network, works great! Until it doesn’t. The problem could be that the adapters can find each other, or that the connection is unstable.

Here are some troubleshooting tips that could help with stabilizing the connection:
– Set the network name to some custom name, using the Utilities software which comes with the adapters. Don’t use the default “HomePlugAV”. Check whether there are unknown network adapters which use the name “HomePlugAV” on your network. This could indicate Neighboring Networks Interference.
– Plug the adapter directly into the wall socket. Extension cords degrade the signal.
– Buy adapters with built-in power sockets, and some form of filter. These help to reduce interference even further.

You should see improved performance with these tips

When an application isn’t visible in WebSphere

Every once in a while, WebSphere doesn’t like you. You try to install or update an application, but it refuses to do so, saying the application already exists, and you need to specify a different name. And when you look at the list of your applications, the one you try to install or update isn’t there anymore!

WAS-deployment_error_detailed

Solving this problem isn’t hard, if you know where to look.

1. Shut down the server

2. On the filesystem, remove the application from the following locations:

  • <profile root>/config/cells/<cellname>/applications/<application name>
  • <profile root>/config/cells/<cellname>/blas/<application name>
  • <profile root>/config/cells/<cellname>/cus/<application name>

3. Delete everything from the profile/temp and profile/wstemp directories

4. If needed, edit the following file:
<profile root>/config/cells/<cellname>/nodes/<nodename>/serverindex.xml
Look for the tag <deployedApplications>, and remove your application from there.

5. start the server

You should be able to install your application again.

Sources:
http://www-01.ibm.com/support/docview.wss?uid=swg21590141
http://www.albeesonline.com/blog/2008/04/15/application-already-exists-in-the-configuration-repository-error/

Java: Remove an element from a List

One of the more common tasks in programming is removing a specific element from a list. Although this seems to be straight-forward in Java, it’s a bit more tricky.

Before we start, we should build our list:

   public ArrayList<String> createList(){
      ArrayList<String> myList = new ArrayList<String>();
      
      myList.add("String 1");
      myList.add("String 2");
      myList.add("String 3");
      myList.add("String 4");
      myList.add("String 5");
      
      
      return myList;
   }

Let’s say we want to remove the String “String 2”. The first thing that comes to mind is to loop through the list, until you find the element “String 2”, and remove that element:

   public List<String> removeFromListUsingForEach(List<String> sourceList){
      for(String s : sourceList){
         if (s.equals("String 2")){
            sourceList.remove(s);
         }
      }
      return sourceList;
   }

Unfortunately, in this case, this will throw a

java.util.ConcurrentModificationException

I said “in this case”, because the exception is not always thrown. The details of this strange behavior is out of scope for this blogpost, but can be found here.

There are several ways to remove an element from a list. Depending on your personal preference, and which version of Java you use, here are some examples.

1. Use a for-loop which loops backwards
You can use a for-loop, which runs from the end of the list to the beginning. The reason you want to loop in this direction is, that when you’ve found the element you want to remove, you remove the element at that index. Every element after this one will shift one position towards the beginning of the list. If you’d run the loop forward, you’d have to compensate for this, which just isn’t worth the effort.

   public List<String> removeFromListUsingReversedForLoop(List<String> sourceList){
      for(int i = sourceList.size()-1; i >= 0; i--){
         String s = sourceList.get(i);
         if (s.equals("String 2")){
            sourceList.remove(i);
         }
      }
      return sourceList;
   }

This works in every Java version since 1.2, although you can’t use generics until Java 1.5.

2. Use an Iterator
Another way to remove an element from a list is to use an Iterator. The Iterator will loop through the list, and, if needed, can remove the current element from that list. This is done by calling

Iterator.remove()
   public List<String> removeFromListUsingIterator(List<String> sourceList){
      Iterator<String> iter = sourceList.iterator();
      while (iter.hasNext()){
         if (iter.next().equals("String 2")){
            iter.remove();
         }
      }
      return sourceList;
   }

This works in every Java version since 1.2, although you can’t use generics until Java 1.5.

3. Use Java 8 Streams
What you’re essentially doing here is make a copy of the list, and filter out the unwanted elements.

   public List<String> removeFromListUsingStream(List<String> sourceList){
      List<String> targetList = sourceList.stream()
            .filter(s -> !s.equals("String 2"))
            .collect(Collectors.toList());
      return targetList;
   }

This works since Java 1.8. More about Java 8 can be found here.

Distributed Websearch

When it comes to searching the web, the first thing that comes to mind is Google. Or Bing. Or Yahoo. Sure, it’s easy, fast, and sort of reliable. However, there are a few problems with these kinds of services.

First of all, there’s the filter bubble. These search engines (and social networks) will present results that are specifically tailored for you. On the one hand, this is a good thing, because you’ll probably find what you’re looking for faster. However, you’ll never find information that COULD be relevant, but is filtered out for you. Your world view is being limited by this filter bubble.

You can work around this by using “anonymous” search engines, such as DuckDuckGo or Startpage. These search engines don’t store your personal data, and therefore can not create a filter bubble for you. As for Startpage, that search engine will act as a proxy between you and Google. You will usually get high quality results to your search query.

Another problem is “The Right To Be Forgotten”. The European court ruled that Google needs to provide the option to delete certain search results from its index. When you’ve had problems, which have been resolved, you shouldn’t be judged on the basis of those past problems. However, once information is on the Internet, it’s very hard to remove. The toothpaste is out of the tube, the genie out of the bottle. You’ll never get it back in. Google provides links to this information, it does not provide the information itself. Removing the links is not the same as removing the information. We have seen the same misunderstanding in the case of The Pirate Bay.

YaCy

Since a couple of years, there is YaCy. YaCy is a distributed search engine, that ensures TRUE anonimity, and is impossible to take down. You need to install a piece of software on your computer, browse to the local webserver (probably http://localhost:8090/), and start searching. At this moment, there are 1.7 billion documents in the public network. In contrast, the web is estimated to contain 15 to 55 billion documents.

There are a couple of other interesting features in YaCy, such as creating a search engine for one specific topic, or a search engine for your local intranet. Give it a try.

Ingress

About one week ago, I started playing Ingress, a mobile game that is basically a hybrid between geocaching and capture the flag. And when I say “mobile game”, I mean that you have to walk around with your phone. That is, outside, in the real world.

The game centers around a newly discovered substance called Exotic Matter, or XM, which is associated with a mysterious alien race called the Shapers. This XM leaks into our world through portals, which are real-world objects such as art, architecture, landmarks, and so on. You need to go out and find these portals, and capture them. Once captured, you can link portals together, and create control fields.

There are two factions in the game: the Enlightened and the Resistance. The Enlightened embrace this new phenomenon, and think it is the next step in human evolution. The Resistance disagree: they think the Shapers are dangerous and need to be stopped. When you start playing the game, you need to chose a side. Be careful, you can only change this once!

Upsides of the game:

  • You need to walk around, outside. The American Heart Association recommends walking 10.000 steps per day, on avarage. With this game, that’s peanuts!
  • You’ll explore the city, finding pieces of art that you never knew were there in the first place.
  • As with every group activity, you’ll meet new people. Whether it is because you are invited to a local Ingress community, or because you’re hacking portals and another player is nearby.

Downsides:

  • Since this game is created by google, it probably is a privacy disaster. It needs to send your location directly to google, in order to update the game correctly. However, most of the time you are at those locations because of the game.
  • The game is very demanding of your phone. The graphics, constant Internet connection, and GPS will drain your battery quickly. An extra batterypack is no luxury.
  • I already mentioned the constant Internet connection. This game uses a lot of data. Make sure your mobile data plan is large enough.

The Pomodoro Technique

One of my favorite time management methods is the Pomodoro Technique. The method is basically as follows:

  1. Pick a task.
  2. Work on it for 25 minutes. This is called a Pomodoro.
  3. Take a 5 minute break. Do something totally unrelated to your work.
  4. Work for another Pomodoro, or 25 minutes. This can be on the same task, or a new one if the previous one is finished.
  5. After 4 Pomodoros, take a longer break of 15 to 30 minutes.

Because of the breaks, your mind gets just enough rest to stay focussed. Also, because you’re supposed to work on one task, and one task only during your Pomodoro, the quality of your work can go up.

Ofcourse, the technique is completely customizable. Do you think 25 minutes is too short (or too long)? Try 45 minutes (or 10 minutes). Do you need a long break sooner? Do it every 3 Pomodoros.

No I’ve made my own Pomodoro tracker. It’s written in Java 8, and it’s open source. You can find the source here . A runnable version can be found here.

This is still work in progress, and I mainly made it as a challenge to myself. If you like it, do whatever you want with it.

The pomodoro is running!
The pomodoro is running!

When the screen and the tray icon are red, a pomodoro is running. Right clicking the tray icon will bring up a menu.

Pomodoro break.
Pomodoro break.

When the screen and the tray icon are green, you are on a break.

Waiting for the next Pomodoro to be started.
Waiting for the next Pomodoro to be started.

When the screen and the tray icon are blue, the program is waiting.

Changing the settings
Changing the settings

The settings allow you to specify the location of the program, the times and the number of Pomodoros between long breaks.

Java 8

On 18 march 2014, Oracle launched Java 8. I’ve had a little time to play with it. Here are my first experiences.

Eclipse

Eclipse 4.4 (Luna) will get support for Java 8. However, Eclipse 4.3.2 (Kepler) can support Java 8 by installing a feature patch. This page shows how to install the patch.

Once installed, you’ll need to tell your projects to use java 8. First add the JDK to eclipse:

  • Go to Window -> Preferences
  • Go to Java -> Installed JREs
  • Add Standard VM, and point to the location of the JRE
  • Then go to Compiler
  • Set Compiler compliance level to 1.8

Then tell the project to use JDK 1.8:

  • Go to Project -> preferences
  • Go to Java Compiler
  •  Enable project specific settings
  •  Set Compiler compliance level to 1.8

Now you should be able to develop your applications using Java 8.

Maven

To enable Java 8 in Maven, two things need to be done:

  1. Maven must use JDK 1.8
  2. Your project must be Java 8 compliant.

To tell maven to use JDK 1.8, point the JAVA_HOME variable to the correct location.
For the second point, make the project Java 8 compliant, add the following snippet to you pom.xml file:

<build>
  <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-compiler-plugin</artifactId>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
        </configuration>
      </plugin>
  </plugins>
</build>

Feature: streams

Streams can be used to iterate over collections, possibly in a parallel way. This has the advantage of making use of the multi-core architecture in modern computers. But more importantly, it makes the code shorter and more readable.

Case in point, consider the following code, for getting the minimum and maximum number in an array:

   public void minMax(int[] array){
      int min = array[0], max = array[0];
      for (int i : array) {
         if (i < min) {
            min = i;
         } else {
            if (i > max)
               max = i;
         }
      }
      System.out.println("Max is :" + max);
      System.out.println("Min is :" + min);
   }

Nothing too shocking. But with Java 8 this could be done shorter and easier:

   public void java8(int[] array){
      IntSummaryStatistics stats = 
            IntStream.of(array)
            .summaryStatistics();

      System.out.println("Max is :" + stats.getMax());
      System.out.println("Min is :" + stats.getMin());
   }

This method converts the array to an IntStream, and then collects the statistics of all numbers in that stream into an IntSummaryStatistics object. When testing this with an array of 10.000.000 items, spanning the range of 1.000.000 numbers, the performance is more than 5 times better with the first method though. The first running in 12 ms, the second in 69 ms.

Feature: lambda expressions

The biggest new feature of Java 8 is Lambda Expressions. These are sort of inline methods, and are mostly used in combination with streams. To explain this, let’s take a look at the following pieces of code. This will get all the files ending in “.csv” from a directory.

First, using a FilenameFilter:

      File sourceDir = new File("D:\\Tools");
      List<String> filteredList = Arrays.asList(sourceDir.list(new FilenameFilter(){

         @Override
         public boolean accept(File dir, String name)
         {
            return name.toLowerCase().endsWith(".csv");
         }
         
      }));

Now, using a Lambda:

      File sourceDir = new File("D:\\Tools");
      List<String> filteredList = Arrays.asList(sourceDir.list())
            .stream()
            .filter(s -> s.toLowerCase().endsWith(".csv"))
            .collect(Collectors.toList());

Notice line 4, with the filter command. This replaces the accept method in the FilenameFilter. What it effectively does is the following:

For each String in the stream:
 - assign the String to s
 - Call s.toLowerCase().endsWith(".csv"), this will return a boolean
 - If the result is true, the String is passed to the next method in the stream
 - If the result is false, the next String is evaluated

Using JNA to get the active program on Windows

This question on StackOverflow explains how to get the currently active program on Windows. This means, the program that is in the foreground, and receiving user input. However, there’s a lot going on there that isn’t explained. And the code could use a bit of cleaning up.

The example uses JNA , or Java Native Access. JNA is a way of accessing platform dependent functions, without the development overhead that JNI (Java Native Interface) requires. You’ll need to read through the tutorial to really get going with JNA, since it’s not that easy.

Library mapping

The first thing you need to do to access the native functions, is to map library you want to use. You can do this using an interface, or using a class. In this case, we’ll use a class. The following code will load the Process Status API.

static class Psapi
 {
     static
     {
         Native.register("psapi");
     }
 }

Function mapping

When the library is mapped, you need to map the functions of that particular library you want to use. Depending on the method you chose for mapping the library, this too can be done in an interface or a class. Since we chose the class, the method will be added there.

static class Psapi
{
     //mapping the library is skipped.
     public static native int GetModuleBaseNameW(Pointer hProcess, Pointer hmodule, char[] lpBaseName, int size);
}

Getting the needed functions

The example displays two properties of the active window: the title, and the name of the corresponding process. To get this information, we need the following libraries and functions:

psapi:
GetModuleBaseNameW

user32:
GetWindowThreadProcessId
GetForegroundWindow
GetWindowTextW 

kernel32:
OpenProcess 

In addition, we need some static fields to correctly call kernel32’s OpenProcess:

 public static int PROCESS_QUERY_INFORMATION = 0x0400; //1
 public static int PROCESS_VM_READ = 0x0010; //1

Getting the title of the active window

To get the title of the active window, we need to do the following:

  • Create a buffer
  • Get the active window
  • Read the title to the buffer
  • Convert the contents of the buffer to a String

In code, it looks like this:

private static String getActiveWindowTitle(){
     char[] buffer = new char[MAX_TITLE_LENGTH * 2]; // create buffer
     HWND foregroundWindow = User32DLL.GetForegroundWindow(); // get active window
     User32DLL.GetWindowTextW(foregroundWindow, buffer, MAX_TITLE_LENGTH); // read title into buffer
     String title =  Native.toString(buffer); // convert buffer to String
     return title;
}

Getting the active window process name

Getting the name of the process is a bit more complicated. To do this, we need the following steps:

  • Create a buffer
  • Create a pointer
  • Get the active window
  • Get a reference to the process ID
  • Get a reference to the process
  • Read the name of the process to the buffer
  • Convert the contents of the buffer to a String

In code, it looks like this:

private static String getActiveWindowProcess(){
     char[] buffer = new char[MAX_TITLE_LENGTH * 2]; // create buffer
     PointerByReference pointer = new PointerByReference(); // create pointer
     HWND foregroundWindow = User32DLL.GetForegroundWindow(); // get active window
     User32DLL.GetWindowThreadProcessId(foregroundWindow, pointer); // Get a reference to the process ID
     Pointer process = Kernel32.OpenProcess(Kernel32.PROCESS_QUERY_INFORMATION | Kernel32.PROCESS_VM_READ, false, pointer.getValue()); // get a reference to the process
     Psapi.GetModuleBaseNameW(process, null, buffer, MAX_TITLE_LENGTH); // read the name of the process into buffer
     String processName = Native.toString(buffer); // convert buffer to String
     return processName;
}

Full code

The complete program will check every second which window is in the foreground, and reports any changes. It will also display how long the window was in the foreground.

import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.platform.win32.WinDef.HWND;
import com.sun.jna.ptr.PointerByReference;

public class EnumerateWindows 
{
     private static final int MAX_TITLE_LENGTH = 1024;

     public static void main(String[] args) throws Exception
     {
          String lastTitle = "none";
          String lastProcess = "none";
          long lastChange = System.currentTimeMillis();

          while (true)
          {
               String currentTitle = getActiveWindowTitle();
               String currentProcess = getActiveWindowProcess();
               if (!lastTitle.equals(currentTitle))
               {
                    long change = System.currentTimeMillis();
                    long time = (change - lastChange) / 1000;
                    lastChange = change;
                    System.out.println("Change! Last title: " + lastTitle + " lastProcess: " + lastProcess + " time: " + time + " seconds");
                    lastTitle = currentTitle;
                    lastProcess = currentProcess;
               }
               try
               {
                    Thread.sleep(1000);
               }
               catch (InterruptedException ex)
               {
                    // ignore
               }
          }
     }

     private static String getActiveWindowTitle()
     {
          char[] buffer = new char[MAX_TITLE_LENGTH * 2];
          HWND foregroundWindow = User32DLL.GetForegroundWindow();
          User32DLL.GetWindowTextW(foregroundWindow, buffer, MAX_TITLE_LENGTH);
          String title = Native.toString(buffer);
          return title;
     }

     private static String getActiveWindowProcess()
     {
          char[] buffer = new char[MAX_TITLE_LENGTH * 2];
          PointerByReference pointer = new PointerByReference();
          HWND foregroundWindow = User32DLL.GetForegroundWindow();
          User32DLL.GetWindowThreadProcessId(foregroundWindow, pointer);
          Pointer process = Kernel32.OpenProcess(Kernel32.PROCESS_QUERY_INFORMATION | Kernel32.PROCESS_VM_READ, false, pointer.getValue());
          Psapi.GetModuleBaseNameW(process, null, buffer, MAX_TITLE_LENGTH);
          String processName = Native.toString(buffer);
          return processName;
     }

     static class Psapi
     {
          static
          {
               Native.register("psapi");
          }

          public static native int GetModuleBaseNameW(Pointer hProcess, Pointer hmodule, char[] lpBaseName, int size);
   }

     static class Kernel32
     {
          static
          {
               Native.register("kernel32");
          }

          public static int PROCESS_QUERY_INFORMATION = 0x0400;
          public static int PROCESS_VM_READ = 0x0010;

          public static native Pointer OpenProcess(int dwDesiredAccess, boolean bInheritHandle, Pointer pointer);
     }

     static class User32DLL
     {
          static
          {
               Native.register("user32");
          }

          public static native int GetWindowThreadProcessId(HWND hWnd, PointerByReference pref);
          public static native HWND GetForegroundWindow();
          public static native int GetWindowTextW(HWND hWnd, char[] lpString, int nMaxCount);
     }
}

JFreeChart

JFreeChart is an open source Java library for creating charts. The project started in February 2000, and is currently the most widely used Java charting library. It offers a large number of charts, examples of which can be found here.

To get started with JFreeChart in your maven project, add the following to your pom.xml

<dependency>
<groupId>org.jfree</groupId>
<artifactId>jfreechart</artifactId>
<version>1.0.17</version>
</dependency>

First, you need to create a Dataset to display in your chart. One of the most used Datasets by the library is the CategoryDataset, so let’s focus on that one.

DefaultCategoryDataset set = new DefaultCategoryDataset();
for (int i = 0; i < records.size(); i++){
Record record = records.get(i);
set.addValue(record.getCorrect(), // y-value
record.getStage(), // dataset
Integer.valueOf(i)); // x-value
}

The parameters for the addValue method are as follows:

  1. The actual value of this record
  2. The row key. When adding multiple sets to the same chart, so you can compare them, use different values for this parameter
  3. The column key, or the column at which this value will appear. Since this must implement the Comparable interface, it will be sorted.

Once you have the dataset, you can use it to create  the chart itself.

JFreeChart chart = ChartFactory
.createLineChart("Score progress over time", // title
"Iteration", // category axis label
"Score", // value axis label
set); // dataset

With this dataset, you can create a number of charts. In this case, a line chart will be created, but you can also create an area chart, a bar chart, a stacked area / bar chart, and a 3D line / bar chart.

The library provides components to display the chart on screen, and to write it to file. The latter option involves only a couple of lines of code.

File output = new File(WORK_DIR, "chart.png");
try {
ChartUtilities.saveChartAsJPEG(output, // file
chart, // chart
800, // width
600); // height
} catch (IOException e) {
e.printStackTrace();
}

And when everything is ok, you’ll end up with something like this:chart

Downloading an entire podcast archive

These days I spend way too much time in my car. And I’d like to make that time at least a bit more useful. So I listen to podcasts. However, downloading all these podcasts can also take a lot of time, especially when you want to download an entire archive. Using Linux, it only takes one simple little script to do it.

for i in $( curl -s <a href="http://www.nature.com/nature/podcast/archive.html">http://www.nature.com/nature/podcast/archive.html</a> |
grep -oie "http.*\.mp3" |
sort |
uniq );
do
echo "${i}:" ;
curl -o ${i##*/} -# ${i};
done

Download

Ok, so it isn’t that simple. Here’s a breakdown of what it does:

for i in $( <list> ); do <commands> ; done

This is the basic for-each loop. For each item in <list>, do <command>.

curl -s <a href="http://www.nature.com/nature/podcast/archive.html\">http://www.nature.com/nature/podcast/archive.html</a>

Retrieve the page (in this case http://www.nature.com/nature/podcast/archive.html) silently. However, it will still output the page to the standard output stream, which is what we want.

grep -oie "http.*\.mp3"

Match the pattern starting with “http”, and ending with “.mp3”. The -o switch will only show the part matching the pattern, the -i will ignore the case, and the -e indicates the pattern will follow.

sort | uniq

These two are combined, because they are basically self explanatory: “sort” sorts the list, “uniq” will remove duplicates from the list.

echo "${i}:"

This just prints the URL of the file to retrieve.

curl -o ${i##*/} -# ${i}

This will retrieve the URL, specified by ‘${i}’. The ‘-#’ means it will display a progress bar. And lastly, it will write the retrieved file to disk. The filename, indicated by ‘-o’, will be the same as the last part of the URL. This means ‘http://www.nature.com/multimedia/podcast/nature/v503/n7477/nature-2013-11-28.mp3′ will be translated to ‘nature-201podcasts3-11-28.mp3

And the end result will be about 400 podcast episodes to listen to. That will keep you busy for a while…