Java Realtime Performance

I’m working with Java project that requires very advanced manipulations of images. In fact, I’m doing most of the manipulation using OpenCV, and I’m using JNI to wrap around the OpenCV functions that I need. I am extremely satisfied with the performance OpenCV gives, the people who wrote the OpenCV code deserve great great credit for the code. In sharp contrast to what I experience with the code Java devs wrote.

I started out optimistic over the choice of my programming language, my first working iteration of the project works fine, but its performance is nowhere near to realtime (getting about 1 frame per 2 seconds.) I’ve done some optimizations of MY code and its helped a lot. I’ve been able to push the frame rate up to about 10-20 frames per second, which is great, but what I’m finding is that to do any further optimizations I have to rewrite Java code to do the same thing but 10-20x more efficient.

I’m appalled at how the developers of Java pay very little attention to performance, especially when writing the classes for Media related classes. I’ve downloaded OpenJDK and I’m exploring the functions I’m using. For example, there is a function under the Raster class called getPixels(…) and it gets the pixels of the image. I was expecting this function to be a highly optimized function in the source code, with several calls to System.arrayCopy to further optimize performance. Instead what I found was extremely “Classy” code, where they are calling 5-6 different classes and 10-20 different methods just to accomplish what I can do in one line:

for (int i =0; i < n; i++) {
  long p = rawFrame[i];
  p = (p << 32) >>> 32;
  byte red = (byte) ((p >> 16) & 0xff);
  byte green = (byte) ((p >> 8) & 0xff);
  byte blue = (byte) ((p) & 0xff);
  byte val = (byte)(0.212671f * red + 0.715160f * green + 0.072169f * blue);
  data[i] = val;
  grayFrameData[i] = (val & 0x80) + (val & (0x7f)); 
}

The code above transforms an image to grayscale and gets the float pixel data, in roughly 1-10ms. If I wanted to do the same with Java built in functions, the conversion to grayscale itself takes 200-300ms and then grabbing the float pixels takes about 50-100ms. This is unacceptable for real time performance. Note to get a speedup, I make heavy use of bitwise operators, which Java devs shy away from.

I understand that they need to handle the general case, but even so, can’t they at least give options for optimizations or at the very least a warning how slow this code may perform.

My question is, at this late point in the development (I already have my first iteration, not I’m working on a second that performs more in real time) should I bite the bullet and switch over to C/C++ where I can fine tune things a lot more, or should I stick with Java and hope things will become more realtime friendly so that I won’t have to rewrite already implemented Java code to get a speedup.

I’m really beginning to become disgusted with how “classy” and slow Java really is. The amount of classes there are seems like overkill.

Realtime communication java to javascript

I have a java program which collects realtime sensor data from a COM port and I want to send it to javascript without using node.js. The sensor data should be givin arround at least 2 times a second.

Sun Java Realtime System on VirtualMachine / cloud

Just wondering if anybody can run/compile application for Sun Java Realtime system JRE on a VM such as VMWare or on the Cloud such as on Amazon EC2 ? I know it is not ideal running Realtime java on a

Producer-consumer architecture with Java RealTime

I am working on designing trade system using Java Realtime ( Sun JRTS 2.2 ) and would like to few questions about best practices, because I am afraid of inventing the wheel and pretty sure that my tas

Realtime API Java suport and examples

I’m trying to use realtime api in java. I have found many examples in javascript but none in java. Is java supported and if not how i’m i going to implement my java app using the functionality of real

High-performance realtime data display

I’m trying to find a tool for plotting data (mostly line graphs and such) that can be used for high performance applications. My data window typically contains between 500 to several thousand points,

Realtime MessageConsole GUI in Java

I have to insert a GUI Console in my Java application. I have chosen MessageConsole.java class to do this. In my app I use this code to use the class : logPanel = new JPanel(); JTextArea textArea = ne

Java Performance measurement

I am doing some Java performance comparison between my classes, and wondering if there is some sort of Java Performance Framework to make writing performance measurement code easier? I.e, what I am do

Java performance on OpenVMS

I have try to test Java performance on OpenVMS (Itanium, OS 8.3) So I create some sample class to test performance as below HEZ[AUN]$type Test.java import java.util.Date; public class Test { public st

java performance : string literal

I would like to know about the relation between string literal and java performance. for example, uses of below statements number of times makes any impact in performance.I have thousands of classes a

monitor Java proccess memory in realtime

I have a java application that its memory starting to jump and fall after few days. Is there a tool that can show me the variables/members sizes during run/debug in real time? Debugging it with eclip

Answers

My suggestion would be it depends on how important the manipulation of images is compared to the project as a whole, and relative to whatever advantages java does bring. Clearly you can write fast code in java (as you demonstrated) if you need to. However, if 80% of your project is going to consist of such optimization, I would certainly rethink Java as the language choice here.

On the other hand, if this represents 20% of the application, with the other 80% being the user functionality around providing this conversion, then perhaps having to do the work to get the manipulation done is a worthwhile trade off to not have to deal with your own memory management, and to have whatever other APIs java is giving you for user interaction (Web, Swing, SWT, whatever you are using).

Java is not known for its real time abilities due to the garbage collector. That might come to bite you as well, so be careful about that.

My question is, at this late point in the development (I already have my first iteration, not I’m working on a second that performs more in real time) should I bite the bullet and switch over to C/C++ where I can fine tune things a lot more, or should I stick with Java and hope things will become more realtime friendly so that I won’t have to rewrite already implemented Java code to get a speedup.

You are asking should I

  1. Switch to a language where I can satisfy my performance requirements.
  2. Stick with Java and hope things improve.

There might be other options…. but option 2 doesn’t seem realistic, you can’t just “hope” that the code becomes faster :p

A few points to note:

  1. OpenJDK does not nescceraly have the same performance as the Sun JDK, have you tried the Sun JDK?
  2. If the performance optimizations that you need done are in a few methods, then it might be worth re-writing them and sticking with Java…

Premature optimization is the root of all evil.

Rather than complain, write an optimized set of libraries and release them, but it would just be wrong to create a “Reference” java implementation that was pre-optimized to some non-existent target.

The main point of a reference implementation is to make understandable, maintainable code–it has to be. I think that there was always an expectation that where necessary vendors would analyze this understandable version and re-implement parts for speed.

In addition to what others have said, you can contribute optimizations to the JDK. If you can provide a powerful optimization that doesn’t sacrifice generality or readability, I expect you’ll be able to get your patch included in a future JDK release.

Thus, you don’t have to hope the JDK can become better. You can help make it happen.

As I understand it, the very latest versions of Java (or maybe it was JavaFX) have methods that allow you to access advanced functionality in the Video hardware of your system. I’m sorry I am being so general, I believe I heard about it on Java Posse and since I’m stuck in Java 1.3 land I never really had a chance to check it out–but I do remember hearing something like that.

Here’s something about it: But it looks like it will only be in Java 7 🙁

Also looks like it will only support playing a stream and rudimentary stream manipulation at first–but maybe the “Wait around and Java will improve” approach might actually work.

What’s stopping you from writing an optimized version of the methods you wish to use instead of using the built-in methods? If that’s not possible, why not write your objects in a more native language, and import that into your existing application?

I don’t know how much of a performance boost you’ll get, but if you have a long-running process doing repetitive things you should try to run the Server Hotspot VM using java -server . It performs much better than the client VM that’s the default on Windows, which is optimized for fast startup time.

I’ve done computer vision work with Java, and I may get downvoted for saying this, but it is perfectly usable for computer vision and realtime stuff, you just have to know how to use it.

Potential Optimizations:

If you need help optimizing your code, I’d be glad to assist — for example, I can tell you that you will probably get a performance boost by making a method

`public static final int getGrayScale(final int pixelRGB){
    return (0.212671f * ((pixelRGB >> 16) & 0xff) + 0.715160f * ((pixelRGB >> 8) & 0xff) + 0.072169f * ((pixelRGB) & 0xff));
}`

and using this in your for{pixels} loop. By using a method call, the JVM can much more heavily optimize this operation, and can probably optimize the for loop more too.

If you’ve got RAM to burn, you can create a static, final lookup table of output grayscale bytes for all possible 24-bit pixel pixel colors. This will be ~16 MB in RAM, but then you don’t have to do any floating point arithmetic, just a single array access. This may be faster, depending on which JVM you are using, and whether or not it can optimize out array bounds checking.

Places to find similar, faster image processing code:

I would strongly suggest that you take a look at the code for the ImageJ image processing app (can’t link thanks to StackOverflow being retarded) & its libraries, specifically ij.process.TypeConverter. Just like your code, it relies heavily on direct array operations with bit-twiddling and a minimum of extra array creation. The Java2D libraries (part of the standard JRE) and the Java Advanced Imaging(JAI) library (can’t link thanks to StackOverflow being retarded) provide other ways to do image processing directly on image data rapidly without having to roll your own operation every time. For Java2D, you just have to be careful which functions you use.

Why the Java2D libraries are so indirect:

Most of the “class-iness” is due to supporting multiple color models and storage formats (I.E. HSB images, float-based color models, indexed color models). The indirection exists for a reason, and sometimes actually boosts performance — the BufferedImage class (for example) hooks directly into graphics memory in recent VMs to make some operations MUCH faster. Indirection lets it mask this from the user a lot of the time.

It is not clear that you are really asking about realtime. There is a difference between realtime and real fast. For real fast, it is sufficient to consider the average case behavior. Throughput is the main concern. Realtime means be able to finish some task within a fixed amount of time each and every time. Or course, there are applications that need both.

In a conventional Java implementation, such as OpenJDK, the garbage collector is the biggest problem for attaining realtime behavior. This is because the garbage collector can interrupt the program at any point to do its work. My company, aicas, has implementation of Java that does not require a separate thread for garbage collection. Instead, a bit of GC work is done at allocation time. Effectively, allocation is payed for by marking or sweeping a few blocks for each block freed. This has required a full reimplemenation of the virtual machine.

Compilation is another point where realtime Java differs from conventional Java implementations. Realtime Java technology tends to use static or Ahead-of-Time (AoT) compilation instead of JIT compilation. JiT may be okay for your application, as you may be able to tolerate the “warm up” time required by a conventional VM to compile the most used classes. If this is so, than you probably do not have realtime requirements, just throughput ones.

If you are interested in ensuring that frame decoding is not interrupted by garbage collection, then it would make sense to use a realtime implementation of Java and perhaps AoT compilation as well. The Real-Time Specification for Java (RTSJ) also provides other support for realtime and embedded programming such as RelatimeThread, AsyncEventHandler, and RawMemoryAccess.

Of course, obtaining good performance, whether realtime or real fast, requires attention to details. The over use of temporary object is not helpful. Allocation always entails extra cost, so should be minimized. This is a major challenge for functional languages, which do not allow changing the state of object. However, one should take care to understand the critical paths of the code being written to avoid unnecessary optimizations. Profiling is essential for understanding where optimization effort is best spent.