java performance

[sala]

hi all

just a quick question. I am a switcher -) and a java developer. I just updated to the new 1.4.1 and made a quick test of performance. The results are quite shocking. Consider the following simple class (it's not PHP but the coloring is just fine):

import java.util.*;

class speedtest {
	
	public static void main(String args[]) {
		long start = System.currentTimeMillis();
		Random r = new Random();
		for (int u = 0; u < 10000000; u++) {
			double i = r.nextDouble();
			double j = r.nextDouble();
			double ij = i/j;
			}
		long stop = System.currentTimeMillis();
		System.out.println("LOOP NEEDED "+(stop-start)+ "MS.");
		}
		
	}

On my old wintel (700mHz, 256m ram) it takes about 3-4 secs.

On my new 1GHz (512m ram) powerbook it takes about 10-11 secs.

Is there anything I overlooked? Is it really possible that the 1.4.1 implementation is about 3 times slower than on a not even comparable wintel machine?

I am looking forward to hearing about your experience with java performance on macs.

thanks
sala

 

[bousozoku]

If you run the code against JDK 1.3.1, you'll find that it's probably faster. I'm extremely disappointed in the performance overall. When using GUI, some things slow to a crawl. I've noted a few things that are somewhat faster with 1.4.1 than with 1.3.1 though.

However, I believe there is some smart caching involved and that may take time to work itself out. If you've had 1.4.1 on the PC for a while, it's probably feeling pretty good, while on the Mac, it's not quite used to its new home.

I'll try the code when I've got some folding downtime.

 

[bousozoku]

Okay, I ran the test on both of my machines.

The dual G4/800 (not that the dual part matters) got:

3.0419
3.0429
3.0453

It shouldn't increase. It should decrease.

I moved the code to the G3/400 which has 1.3.1 on a fast hard drive. It got:

2.0258
1.9465
1.9465

Recompiling the code with the 1.3.1 compiler had no effect on the G3.

I think Apple has some work to do. There will probably be an update in the next week or two. Then again, I remember reading "The problem is with your code." when something would compile and run on every other C compiler including gcc on other platforms, but not Mac OS X.

 

[khollister]

Well, this is certainly odd...

I compiled the test code and got

On my DP 1.25GHz G4 - 20s !!!!!!
On my P4 1.8GHz w XP Pro - 4s

Never mind the 5:1 difference, why are we not getting consistent results?

Keith

 

[bousozoku]

It looks as though it's going in reverse, the slower the PPC processor, the faster the results.

Could the System.currentTimeMillis(); be returning the wrong value?

 

[khollister]

I think a co-worker & I found the problem - the random number method is pranged. If you use this code:

import java.util.*;

public class Test2
{
public static void main(String args[])
{
long start = System.currentTimeMillis();
double j = 0;
for (int u = 0; u < 10000000; u++)
{
double i = u + j;
j = u + j;
double ij = i / j;
}
long stop = System.currentTimeMillis();
System.out.println("LOOP NEEDED " + (stop - start) + "MS.");
}
}

We get much saner numbers:

PM DP 1.25 GHz:
1.4.1 540 ms
1.3.1 667 ms

iMac G4 700 Mhz:
1.4.1 985 ms
1.3.1 1212 ms

These are the type of results I would expect, 1.4 is faster, and the 1.25 is almost twice as fast as the 700.

The bad news is that it runs in 215 ms on my 1.8 GHz P4 with 1.3.1. I do not have 1.4 installed there since we're using Weblogic 7 at work (doesn't support 1.4 yet - native IO issues). There is an advantage to the P4 with floating point since I'm sure the Java VM is not Altivec optimized, so we're going to tinker up a string mangling test - I would expect the results to be a little closer.

I would expect your 800 DP to get a little better results than the iMac.

I have NO idea what the hell Apple did to the random number generator.

 

[sala]

Thanks a lot for those replies.

Indeed, the Random object slows down the programm quite dramatically. However, I think this lies in the very nature of generating random numbers, and not only in the implementation.

When I run the modified code (without random numbers), it takes 693 ms on my 1GHz Powerbook (1.4.1). Still, my old wintel machine (700 MHz) is about two times faster (344 ms, also on 1.4.1).

Hence, it looks like it is not only the implementation of apple's Random class, but the implementation of the JVM on the mac that makes it at least three times slower.

I know that this is not really a sensible performance test, but it gives me some impression of the Java implementation on macs. It look like I will use macs in the future for daily work and wintels to do the math...

 

[khollister]

You are welcome, but I think you misunderstood my comments

The issue with the random method is not the absolute execution time, but the fact that the performance does not scale with CPU speed as one would expect. With the random number generator, my friends G4 700 iMac was considerably faster than my G4 1.25! Furthermore it appeared that 1.4.1 was slower than 1.3.1 - the float test case showed that is not the case (by our tests on Windows & Solaris boxes, 1.4.1 should be considerably faster). We did another benchmark with stringbuffer manipulation, and obtained results similar to the float test.

Make no mistake - Apple needs to optimize their VM on OS X. Our tests also ran 2-3 times quicker on P4 machines. The problem with the random test is consistency.

 

[sala]

hm, I dont understand... In your earlier post, you wrote:

PM DP 1.25 GHz:
1.4.1 540 ms
1.3.1 667 ms

iMac G4 700 Mhz:
1.4.1 985 ms
1.3.1 1212 ms

and now you say

my friends G4 700 iMac was considerably faster than my G4 1.25

Please clarify.

Thanks & regards
Sala

 

[lmalave]

Well, I don't think a tight loop like the example you have given is necessarily an effective benchmark of real-world performance.

That being said, all the benchmarks I've read show that Java performs best on Windows and Linux. See the industry-standard Volano benchmark:

http://www.volano.com/report/index.html

As you can see, Java on Linux and Windows is fast, on Solaris it is significantly slower, and on FreeBSD it is even slower than that. Mac OS X probably comes in somewhere around the BSD/Solaris range. But have faith - just 3 years ago Java on Linux was probably as slow as it is on OS X now. And Apple is fully committed to Java, so I would say within about a couple years OS X will be right up there with Linux and Windows for Java performance.

I use Java all the time, though, and Java on OS X is great. I mostly do server-side stuff, but even using GUI tools I haven't had a problem. I use Sybase jSQL, a pure Java app, which is fast even when dealing with large data sets. But then you have apps like LimeWire which are unbearably slow. But then again LimeWire sucks on Windows too, which just goes to show you that the slowness is usually in the application and not in the Java VM itself.

Don't worry about it, you'll be fine. I mean, Java on Solaris is slow (see benchmarks above), but that hasn't stopped me from deploying mission-critical Java apps on Sun machines. And in your case I assume you'd just be using a Mac for development, not as a deployment platform, so you should be quite happy. I personally like doing development on a Unix machine. I personally like using just plain old Emacs for Java coding, especially with its CVS integration and all