Growth Issues,SN Java BufferedImage Transition

With my SN Project slowly coming closer to completion, I decided that I would try out making a simple game to see what needs work, however like always, my plans where derailed quite quickly. I had a small library of images that was working quite well at the time, however after increasing the library to a couple thousand images, I quickly noticed that things where not working quite right, well… actually not at all, it was throwing a java.lang.OutOfMemoryError and crashing. The memory error that as an easy fix, adding a VMOptions tag to the info.plist with -Xmx1024m allowed for me to set the max memory to a much greater amount, however this was nothing more than a temporary fix, I knew there was a much greater issue behind this problem.

The size of the image library was around 150Mb, which when loaded to memory would be larger due to it being uncompressed, however it should not have been much larger than 2X the base size or 300Mb as an upper limit. I was shocked when memory needed skyrocketed to well over 700 Mb, whereas the standard max for the Java VM is around 100Mb. So I went looking for a memory leak. Java handles most memory issues, but it still can have problems with large collections of data gathered in a short period of time, due to the built in garbage collection running only occasionally as needed. After doing some research via Google, I stumbled upon a complaint about the MediaTracker keeping references to the images it tracks, which while in small numbers is not an issue, but can quickly build up as more images are tracked. This was exactly what I was doing wrong.

The standard Image class in java does not load the image into memory right away, instead it acts as a reference until it is needed. This behavior could be seen when removing the method calls to add images to a MediaTracker, the memory used would only increase to around 50Mb and slowly grow as images where loaded when needed. The problem with this is that it causes an issues with flickering in animations. When each frame is used for the first time, it actually draws it on the second call after the first call forces it to be loaded into memory. The standard practice at the time of Java 1.3 to 1.4 was to use a MediaTracker to force the images to load and for my initial use worked quite well. However my project has finally grown beyond the simple use of a MediaTracker. The next step I took was to try some suggestions on limiting the memory retention by removing images from the tracker when they are loaded, or by using separate trackers for each image as well as forcing garbage collection with System.gc(). A mix of these solutions did lower my memory use, however as usual the trade off for space was time and this setup slowed my image loading algorithm to a crawl.

It was quite apparent that I needed something better, and after a bit more research I decided to transition my SN Project from using the base Image class to the improved BufferedImage class available in Java 1.5+. I had actually been using the BufferedImage class for a few things already, but a total transition was not a simple matter. For the most part you can use BufferedImage anywhere you are already using Image, due to the BufferedImage extending the Image class. One huge benefit was that the ImageIO class, instead of the awt Toolkit, loaded the whole image to memory so a MediaTracker is not needed and this sped things up greatly and removed my memory leak. Compared to before, now it only was using around 230Mb, which was well inside my expected limits.

The big problem that I ran into while changing over my code was that the sneaky way I was loading my animation class into my image array on the engine side would no longer be feasible. Originally, I had found a nifty solution to my space problem, with a bit of tweaking my animation class could extend the Image class and then be inserted into my image array for quickly accessing both the game animations and images through one simple method call. It also only needed minimal conditional checks which I already had in place. While this worked well with Image, there is no such luck with BufferedImage. You can serialize a class that extends BufferedImage, but you cannot deserialize it, because as it lacks the "no argument" constructor needed to reconstruct the base class. This left me frustrated and quite annoyed, by this point the editor was ready and working, but the engine would not accept the image data.

A few hours later, I had separated the animations from the image array and while I was sad that I had to abandon my unique solution, it was probably for the best as it is easier to figure out what the code is doing now. This is where big problem two popped up. I was using a PixelGrabber to export my images to an int array and then reconstructing them with MemoryImageSource and the awt Toolkit to load them back into an Image and then forcing them into memory with a MediaTracker. This was not going to work as I was now avoiding the Toolkit and MediaTracker classes. With some more research I finally pieced together that I could do a similar process but this time to a byte array. A simple example of what I am doing is listed below.

Exporting is the same as saving the image, but to a byte stream instead of a file.

ByteArrayOutputStream bstream = new ByteArrayOutputStream();
ImageIO.write(img, "png", bstream);
byte bytearray[] = bstream.toByteArray();

To convert the image back you can read the byte array using a ByteArrayInputStream.

BufferedImage image = ImageIO.read(new ByteArrayInputStream(bytearray));

At this point, around ten hours after I started, I now had a working engine and editor again, however it was rendering slower than before. I commented out a most of the drawing and logic method calls and found that the engine would run around 50 FPS with minimal drawing, on the other hand, with all calls back on it ran around 8 FPS. Interestingly enough, the logic was the issue, instead drawing a single large transparent image of the GUI was causing a 40 FPS reduction. I was aware that transparency always causes a speed reduction due to increased processes needed to render it, but not by that much!

It took some more searching to figure out what was wrong. In Java images come in a few different types running in different modes, examples of these would be Image, VolatileImage and BufferedImage. I found VolatileImage quite fascinating as it is the fastest since it is always stored in the graphics hardware memory, but the trade off is that it always may or may not be available due to the possibility of it being overwritten by something else in the limited space of video memory. You must repeatedly check to see if it is still there, the descriptions of this made me think of trying to arrange a large group of very hyper children into a pattern, but at any point they may scatter. Luckily, with new changes in the BufferedImage class it too tries to run in video memory if possible, but only if it is setup to be compatible with the current video configuration. Apparently, using the ImageIO read method does not always create the fastest images, it was suggested that you create a more compatible image using the current graphics configuration and draw the image onto that. This also seems to only cause a minimal increase in processing that is easily outweighed by the huge increase of rendering speed. After converting all the SNEngine's BufferedImage objects with the GraphicsConfiguration createCompatibleImage() method a huge increase could be seen, going from 8FPS to around 35FPS, my target being 30FPS.

I still have some more work to do on increasing efficiency, but this tedious transition was an eye opening experience. As usual I have learned more than I expected, but this new information on Java graphics will probably come in handy later. (NOTE: This was found to be true on OSX 10.6 compiling for Java 1.5+, while most things would be similar on other platforms, the low level hardware acceleration for graphics does slightly differ on each operation system and JDK.)

Java JNI Custom About Box

It is silly just how long it took me to figure out this one line of code... A few months ago, I had finished porting over my SN Project from standard Java to then new JNI Library in Xcode 3.2. However, no matter how much I poked and prodded and searched, I could not figure out how to override the simple About Box I am assuming is provided by the JavaApplicationStub.

Luckily, today I finally stumbled on the answer in an obscure thread on the Apple Mailing List. The short answer is that I was not telling the event that it was already handled before letting the method end, the default box was called as a result of this.

You can see below that you need to add setHandled(true); to the handleAbout ApplicationEvent that is in the OSXAppAdaptor class provided with the JNI Template, or where ever you are handling the EAWT action calls.

public void handleAbout(final ApplicationEvent e)
{
   e.setHandled(true);

I actually feel a bit stupid for not noticing this sooner as a few of the other action call methods already are setting handleAbout but it was not that clear what it was doing so I completely over looked it. Well, with this fixed I now have all the kinks worked out, now on to finishing up the features and finally hitting a stable 1.0 release, though it may take another year.

Simple Java XML Parser

To continue my SN Project work I started to convert the simple object to data save format I had been using to save time during the semester into a xml based file system. This way when I update code it will not break saved game file due to class def not found exceptions. However I quickly ran into an issue that Java did not have a "simple" built in class to handle XML parsing and other free libraries where a little more complex then I was looking for, so although I usually try not to reinvent the wheel while programing, this time I wanted to try my hand at writing a simple Java XML parser.

What this basic XML parser does:  It looks for the starting tag < and then starts appending the tag characters to the node tag.  Then it hits > telling that the tag had ended and the data starts finally it looks for < for the next node.  There is also the case that another < is found in the data which indicates that a nested tag was found.  The older node is pushed on a stack and the parser moves up one child node and begins the tag and data read once more. This is repeated until no nodes are left on the stack indicating the root closing tag has been reached.

As a note:  I have removed most of the advanced checking and my custom xml build methods for security reasons, so you are on your own to add try/catch as needed.  Also, as I am not using any attributes in tags, so this parser does not read them.

Overall the code works well for the xml documents I am reading in, though for one class I may still look into a faster c based solution in the future, since I am working with a JNI library anyway.

You can check out the code HERE
As usual the code is free to use, but please give me some credit if it is used in a large project, or leave a comment about what it was used in.

Tomcat install on OS X 10.4

I had seen Tomcat on job applications before, however this was the first time I had actually played with it. What is it? "Apache Tomcat is an open source software implementation of the Java Servlet and JavaServer Pages technologies. The Java Servlet and JavaServer Pages specifications are developed under the Java Community Process." So it allows you to run Servlets and JSP on a server. Now I needed to find out how to install it on my Lazurus server running OS X 10.4.11 with the latest Java update.

1. First download tomcat from HERE (I used 5.5, so not sure about 6)

2. Then rename the unarchived tomcat folder to "Tomcat"

3. Drop the folder into /Library/

4. Fix the UNIX permissions recursively to 777 (BatchMod is a simple way if you do not know chmod, just check all of the boxes)

5. Edit the /Library/Tomcat/conf/tomcat-users.xml file, see below for example.

 <?xml version='1.0' encoding='utf-8'?>
 <tomcat-users>
  <role rolename="tomcat"/>
  <role rolename="role1"/>
  <role rolename="manager"/>
  <role rolename="admin"/>
  <user username="tomcat" password="tomcat" roles="tomcat,admin,manager"/>
  <user username="both" password="tomcat" roles="tomcat,role1"/>
  <user username="role1" password="tomcat" roles="role1"/>
 </tomcat-users>

6. Start tomcat by using the terminal to run "cd /Library/Tomcat/bin" and then "./startup.sh"

7. In a web browser head to http://127.0.0.1:8080/ and you should see the Tomcat start up page.

If you want to read on further I figured out most of my instructions from the Tomcat Wiki, so head there for more info or if you get stuck.

Just in case you where wondering, as I was, put your jsp and servlet files in the /Library/Tomcat/webaps/ folder. The webaps/ROOT folder being the one displayed at the base url address. So for now I am just using that folder for testing.

JSP is quite simple to pickup at a basic level if you already know Java. However there are a few differences do to it being web based, but so far I like not having to compile manually. I had no problem writing a few simple classes with outputs and will later play with post and get. HAHA, tri-fecto of web programing achieved (PHP, ASP, JSP), hm still need to sit down with Perl one of these days.

JamBot

A while ago I was introduced to the online web game Jam Legend, so I checked out the site. The premiss is the same as any of the music games, you hit the notes as the move on the screen. I played for a while, then got bored and forgot about it till this week. However, after playing it again, and getting bored again, I decided to create my own challenge and finally check out the Robot class in java to see if I could create a program that could play it for me.

The Robot class is a java package that is designed to aid in computer automation. You can have it look at the current screen object and then move the mouse or hit keys. The big challenge for me was getting the bot to be able to recognize what was happening on screen. Like any robotics programing it is easy to say that it will look here and do something, but actually making it happen is quite a bit more challenging. To start off JamBot requires you to drag the transparent window over the game, this sets up a view port for the bot to look at a few different locations, namely in and just above the five note zones. The challenge with this is that when playing not only do the note locations change when you need to hit a key, it also changes the background colors as your multiplier increases. In order to adjust for this, the easiest way is to check the color percent of the note color for that spot, vs just seeing if the color has changed. If you look at the included code, the values have been adjusted to what I though worked best, but you may have better luck with some further adjustments. The next step was to also have the bot look just above the note zone to see if there is a hold note active. Then if a color match is found, instead of unlocking the key, it waits for the hold to finish. After figuring out all this, and a few hours later, I had a working bot, just for the insane tap difficulty which would get around a 90% on most songs. Is it perfect, NO, however it looks and act like a normal person playing with normal skill would.

Now if I was interested in further development I would look into adding separate Robot objects for the different notes, I think this will improve accuracy and timing, by using different threads for each zone. Right now the bot is missing notes due to the main loop not being fast enough to either see the note coming or hit it before it is out of range. I am reasonably sure that a more streamed lined threaded setup will fix this. Also it is missing the ability to select different difficulty setting, and needs an included feature for strum. All which would not be too hard to add, but I will leave that up to some one else, as I have other projects to start.

Quick Instructions for Use
[Only works for Insane Tap]
1. Open JamLegend and get to the point that you can hit play to start the song.
2. Start JamBot and drag window till the lines and circles line up on screen.
3. Click the Setup Button and then drag the window to the side.
4. Once out of the browser window click start.
5. Start your song and watch as it plays its self.

As usual I have uploaded the compiled class (MAC) and the java source code for anyone to develop further, however please credit me if you do. Download Here and enjoy.

Applet ProgressBar Flickering Issue

I had recently taken the the time to convert my EpicGame Code from VB to java, so I could make an applet that could be played online. (I was lazy and did not want to make it flash based) However after finishing it and testing on the mac side I though I was finished. Yet, later I started hearing complaints of the game flickering, I was confused as it worked fine for me and as usual I had double buffering setup, which in theory would prevent any flickering. Still, for some reason Window's version Java would flicker uncontrollably. I spent several hours trying different graphics buffering setups and also drawing to separate panels, but with no success.

Luckily I randomly stumbled onto the problem, it was the JProgressBars that I was using. When you update a value in the progress bar it calls the update method that then calls paint, which would happen concurrently mid screen refresh and draw a blank white screen, which was the rouge flickering.

So on Windows do not use JProgressBars with animation as it messes it up, or you need to extend the class and override the setValue method so it does not update the UI. Oh, and of course on the Mac side it all works flawlessly from the start.

Somnium Nexus Site Up

Now that University Physics is over I have to start working again on my SN Project. However, I wanted to make a decient site for it first, other then the one that I had made at school. Googlepages is not perfect, but it will do the job. Probally the cleanest CSS that I have writen yet. Check it out here SN Project. There is a link for the latest build, though that was about a year ago, however it still works. Just remember it is a Mac only pre alpha, so no guarantee on stability and safety. ^_^

Expect more updates on the SN Project in the future, as it is going to be my Senior Project this year.

UPDATE: Brian gave me some constructive critisim on the art, so I added a more worn feel. Also played with the styling a bit, works in FF3 now, see previous post for the fix.

Java Simple Physics

Well with classes over, I wanted to at least do something with the year of University Physics torture I have endured. Thus I present Java Simple Physics. It is just a quick physics example that uses the kinetic and potential relationship to have balls with different gravity constants drop then bounce with no loss of energy. It took a few hours to get right, mostly because I either did not know what I was doing, or was over complicating things. I hope to slowly improve on this concept till I have a VERY simple physics engine. Hey, it can only help to have a physics engine laying around, one day I might actually need one.

import javax.swing.*;
import java.awt.*;

public class Physics extends Thread
{
 private int worldx = 200, worldy = 900, worldtime = 15;
 private JFrame frame = new JFrame();
 private Ball ball,ball1;

    public Physics() 
    {
     // Set up frame
     frame.setSize(worldx,worldy);
     frame.setVisible(true);
     // Make new ball
     ball = new Ball(400,450,9.8,worldx,worldy,worldtime);
     ball1 = new Ball(400,450,3.8,worldx,worldy,worldtime);
    }
    
    public void run()
 {
  while(true)
  {
   try
   {
    // Cover old draw with fresh screen
    frame.getGraphics().fillRect(0,0,worldx,worldy);
    
    // Move and paint ball
    ball.move();
    ball.draw(frame.getGraphics());
    ball1.move();
    ball1.draw(frame.getGraphics());
    Thread.sleep(worldtime);
    
   }
   catch(Exception e){System.out.print(e);}
  }
 }
    
    public static void main(String args[])
    {
     Physics p = new Physics();
     p.start();
    }
}

class Ball
{
 private double gravity = 9.8, t;
 private double x,y,v,a,m,dx,dy,dv,k,u;
 private int worldx,worldy,dir=-1,width = 90;
  
 public Ball(int xin, int yin, double g, int wx,int wy, int time)
 {
  // Set Constants
  worldx = wx; worldy = wy-width; t = time;
  // Reduce gravity to fit with drawing refresh rate
  gravity = g * .02;
  
  // Center ba;; on x axis
  x = worldx/2 + width/2;
  
  // Set varables
  y = yin;
  v = 0;
  a = 0;
  m=.8;
  
  // Preset u and k values based on varables
  k = (m * gravity * (worldy - width)) - (m * gravity * y);
  if(k < 0){ k = 0; }
  u = m * gravity * y;
 }
 
 // Catch and reverse
 public void move()
 {
  // Calculate u and k
  k += u - (m * gravity * y);
  u = m * gravity * y;
  //Find dv based on k = .5mv^2
  dv = ((2 * k) / m) - v;
  
  // Catch and reverse on k or u < 0
  if(k <= 0 && dir == 1)
  {
   dir = -1;
   v = v*(-1);
   a = a*(-1);
   k = 1;
   u = m * gravity * worldy-width/10;
   dv = -1;
  }
  else if(u <= 0 & dir == -1)
  {
   dir = 1;
   v = v*(-1);
   a = a*(-1);
   u = 0;
   k = m * gravity * worldy-width/10;
   dv = ((2 * k) / m);
  }
  
  // Finalise new movement
  y += ((1+dv)*dir)/t;
  
  // Print out values when needed
  //System.out.println("x:"+x+" y:"+y+" u:"+u+" k:"+k+" - dv"+dv*dir);
 }
 
 
 
 public void draw(Graphics g)
 {
  // Draw to screen
  g.setColor(Color.GREEN);
  g.fillOval(worldx - (int)x, worldy - (int)y - width/3, width, width);
 }
 
}

Running Lejos on Mac OS X

One of the classes I took last semester was Artificial Intelligence, and after a long and horrible 9 weeks of prolog bootcamp, we finally hit the robotics part of the class. There was only one problem, we where using Lejos, which is a java based programing language for the Lego NXT bricks. This was fine because I am quite proficient with Java, the problem was that Lejos does not have native Mac OS support, and all of the forums that relate to it only give partial help or promises of support eventually. This is not good enough so I set off on an adventure to get it running on my MacBook Pro.

The first thing I did was to get the latest update to Lejos, big mistake, the linux Lejos 0.4.0beta would not work for me. I finally got it to force build, but this was only enough to get a broken firmware on the brick. Realizing this I went back to the 0.2.0alpha that we where using in class and this compiled fine. You just download it from the Lejos Site and unzip it. Place the entire lejos_nxj folder in the Applications folder. Then open terminal and cd to the build directory aka /Applications/lejos_nxj/build. Then type and run ANT and it will make a build for the Mac OS based on the build.xml file in the build folder.

The problem I ran into is that in terminal you have to ether set up a tcsh shell with the environmental variables set or set them in a .profile file in your home folder. Both of these worked but where annoying and took quite a bit of research on my part to figure out what needed to be done. Also you then need to use a text editor and run the build, compile and link command in the tcsh shell on the .java file each time you run it. Sadly this was no better than the PC solution, which was essentially doing the same things through Eclipse and batch files.

In lab I had done the basic PC Eclipse Setup, but quite frankly the whole button setup made me want to scream, not that was was that hard, but instead I found it very pointless for all of that work that was required. So after I got Lejos working on my Mac I started to write a small application that would let me code and then run the code in one simple step, and at the same time make the PC users in my class envious of its simplicity.

The final result I was quite proud of, and only took about a week of using/upgrading, plus a bit of unix scripting on the back end to get it to the point where I was happy enough to just bring my laptop to lab and use it instead. Also now that the class is over I spent a little time to clean it up a bit more and make it more usable for people other than me. If you are looking for a quick solution or are planing on doing something similar I have uploaded my application, which I am calling LejosTools. Get it Here I hope it can be as helpful to someone else as it was for me. Also I have included the source with it, because I have no plans to continue development. All I ask is if someone uses the code send some credit my way as a developer. Otherwise enjoy...

As a side note, I have not tested other versions of Lejos, but from what I have seen the unix commands have not changed, so if you can get something greater then the 0.2.0alpha working, LejosTools should still work. If they do change just update the lejosfull sh script in the resources folder.