Configuring Eclipse to use a JDK at a location with spaces in it

I was setting up my Eclipse environment with some new plugins today and ran into some trouble, which seems to have bothered many souls as I gather from the posts I found when trying to fix the problem.

Some tools/plugins/features in Eclipse may need to use a JDK rather than a JRE. Sometimes you may even wish to switch between different versions of the JRE or JDK being used since typically 1.4, 1.5/5 and 1.6/6 are all present on a developers’ system at different locations.

I got a mesage from a Maven plugin which said that it needed a JDK to be used to run Eclipse instead of a JRE that it was currently using. Well, this shouldn’t usually present much of a problem. You can choose the virtual machine used to run Eclipse by passing the -vm argument to eclipse.exe. What’s usually needed is something like “eclipse.exe -vm c:\jdk1.6\bin\javaw.exe”. Mind you, javaw.exe is used when you don’t want an associated console window.

I did have JDK 1.6 installed. But it was installed to “C:\Program Files\Java\jdk1.6.0_07″. That, it seems, is the default location. The space in the path caused big problems. I tried quotes\no quotes and other magical incantations to get it to accept the argument with a space but to no avail.

So instead of using the command line switch, I put an entry into eclipse.ini which is in the same directory as eclipse.exe. But this didn’t work either.

Finally, what did work was that I started command.exe (the 16 bit DOS shell) and navigated to the JDK directory using “dir /X” for help which shows names generated in the format “C:\PROGRA~1\JAVA\JDK16~1.0_0\BIN”. Perhaps you remember coming across such filenames.

Anyway, the older 8.3 style directory name worked! My final eclipse.ini is posted below.

-showsplash
org.eclipse.platform
–launcher.XXMaxPermSize
256M
-framework
plugins\org.eclipse.osgi_3.4.0.v20080605-1900.jar
-vm
C:\PROGRA~1\JAVA\JDK16~1.0_0\BIN
-vmargs
-Dosgi.requiredJavaVersion=1.5
-Xms40m
-Xmx512m

I was using Windows XP with SP3 and Eclipse 3.4. There is likely to be a better solution that I missed. If you have it or if this post helped you, please leave a coment.

How to split a file, process the pieces in multiple threads and combine results using a shell script

Say you are in a situation where you have a file with a huge number of records to be processed and the processing of one record does not need data from the processing of previous records (ie. a perfectly paralellizable situation), what can you do to speed up things? Well, here’s what I did when a client recently made a request for statistical data for 300K records instead of his usual request of 20 records as the program that we had earlier made for the purpose wasn’t really made to run fast and wasn’t multi threaded.

Use the “split” command to split a file by number of lines into an appropriate number of files depending upon the configuration of your hardware. Use the “-l” option to specify the number of lines in each file. Then run multiple instances of your program to process the different files in parallel using an “&”. Use “wait” to wait for all background tasks to end. And finally when things are done, merge the different output files together with “cat” in append mode. Voila! Things finish much, much faster.

I used the above steps to make 20 files with 15K lines each since the server I was running the script on was a Sun Solaris 10 T2000 system with 32GB RAM which has an octa-core processor supposedly capable of running 32 threads in parallel. It worked like a charm!

Sample script follows:

split -l 15000 originalFile.txt

for f in x*
do
runDataProcessor $f > $f.out &
done

wait

for k in *.out
do
cat $k >> combinedResult.txt
done

Java Synchronization Benchmark

Here is a very simple and straightforward benchmark to demonstrate how synchronization in Java can affect speed of execution to different extents in Java 1.4 and Java 6.

In the attached image (click above for full size), you can see that for Java 1.4 the synchronized method needs over 700% more time to do its work compared to the non-synchronized method. But for Java 6, the difference is lower with the synchronized method needing only 400% more time to do its work compared to the non-synchronized method.

Remember, JVMs, especially newer ones do some nifty runtime optimizations that most developers are not aware of at all – like “biased locking”. This feature reduces the time taken for reacquiring a lock if the same thread is taking the lock repeatedly. There are other features like “adaptive spinning” and “lock coarsening” as well. Ofcourse, the fact is that it might just be other optimizations that are at work here instead.

Such a simple benchmark does not deserve to have any conclusions drawn from it. Infact the effect may have been greatly amplified here because the program does only an increment within the function so locking becomes the bottleneck. Such a situation may ofcourse be true in “real world” programs as well. But quite often, it is not.

I obtained this result on the single core 2.8Ghz PIV system at my desk at work.

Ideally, one should be using a multi-processor machine and multiple threads to demonstrate such effects. I would expect Java 5 to show results between those for 1.4 and 6.

Don’t post onto your blog too fast to avoid losing traffic!

So I had this bunch of posts that I had written on Orkut about my training at TechM that I recently decided to post to this blog too. I broke up the entire content into several posts and made 14 posts in quick succession (in about an hour which involved formatting, tagging, posting). My advice, don’t do that!

I noticed that since the posts were made so quickly, RSS Feed readers like Google Reader, Technorati etc. seemed to have missed receiving several of the early posts I made in the bunch. From what I can tell, WordPress only seems to offer the last 10 posts to the Feed readers when their bot decides to drop by asking for new posts, if any.

Seems like Technorati didn’t even accept the last 10 posts! I see that they took in only the last 5 posts offered by the WordPress RSS news feed. So that’ll probably hurt me in terms of traffic sent to my blog (which isn’t much to speak of anyway…for now). This would be particularly bad for a blog which has a lot of people subscribing to its RSS feed to stay in the loop.

This could of course be an issue with A-list bloggers like Scoble who make many posts daily….but probably not as quickly as I did for that brief period. But then anyway, the feed readers configure themselves to visit popular blogs more often and, I guess, accept more of their new posts if offered by their RSS feed.

I suppose later someday in a week or two, Google will send its spider/bot to read all of the latest content and then, hopefully, discover the “missed” posts and maybe send some traffic my way. I’ll probably make a new post to index links to all the recent posts to help the spider do so.

I did think about deleting all the posts and then reposting them one by one in a less bursty fashion. But since some of the posts had already been received by feed readers, I decided to spare unsuspecting surfers the possibility of landing up at a page that is no longer around.

So if you ever need to make a bunch of posts to your blog, making 5 posts a day until you run out may be a better idea than making 25 posts in half an hour.