Archive for August, 2008

h1

Running Java in .NET

August 13, 2008

Did you write a java library but now some of your customers would like to use the library in their .NET code? Or do you have a .NET library and you need to integrate with a java application? There is this great tool out there called IKVM (http://www.ikvm.net) – it allows you to cross compile the byte code to the .NET CLR and then run it on top of the .NET framework (swing excluded).

If you have not yet, you have to try this tool. We used it to bridge a java RMI client library to .NET code. So now the .NET application calls the Java RMI client that then calls it’s Java RMI server. Sure, we could have implemented a .NET client to call the server side but this was easy (a days work to play with the tool, get to know it and do the implementation).

Would love to hear from you what you think about IKVM and what you’re experience with it was.

h1

How many objects are too many?

August 11, 2008

Say we are writing a desktop application such as a simple spreadsheet. We could represent each cell as an object and if the cell contains a formula we could implement a listener infrastructure in order to get notifications when a referenced cell is changed.

If we take a 8 x 8 spreadsheet and put a formula in each cell that references 2 other cells we are at 64 cell objects, 128 listeners and probably 64 listener lists. That’s about 256 objects. If we start creating a larger spreadsheet with say 64 x 64 cells that have the same amount of references per cell we are already at 16’384 objects that will be lurking around in the permanent memory. And the worst of all, those objects won’t go away – they are in memory as long as we are using that spreadsheet.

Another design approach could be to store everything in a database. We create a cell table and a cell reference table. Now if we want to display a grid of cells we just query the database and get the data for all cells on the screen. If we cached the calculated value we are now done. If we did not cache the value we have to get each cell that is referenced in the formula and then the value from those cells (and of course continue calculating these cells as well). This can result in a lot of queries to find out what the actual value for a cell should be. In case we are caching the display value we have the same problem when updating a cell. We will have to recalculate all references and their references until we are done.

Both approaches have an issue with referential integrity (somebody has to check that). the first approach is a memory hog while the second approach uses to many queries to calculate the actual value of a cell.

Now what? We have two approaches that both have their drawbacks. Yet excel and open office calc have been around for a while and they both handle such a situation with ease and with a very fast calculation speed. How are they doing it? I played around with those two programs just a little to get a feel for how they may be solving this problem – I create a spreadsheet with a formula that touches as many cells as possible (each formula is dependent on the next one, etc) and then changed the initial value. Turns out open office calc is quite a bit slower than excel. Turns also out that excel guesses the final value and may change it after the calculation is done (create an exponential function, change your value from 1 to 2 where your result changes from say 1 to 4, now change your initial value from 2 to 3, excel will guess the result as 8 and then later correct it to 9).

I made the following two observations after playing with these applications:

  1. Memory usage is low even with larger spreadsheet. Using a file (virtual memory) to store all the data and the references is probably used
  2. If there are no circular references there are cells with 0, 1, 2 … references to other cells. The cells can be seen as a graph that can be processed. One should be able to order the cells in the order of their references and in order of their dependencies. One should then be able to run through all fields from top to bottom and calculate the result (should explain this one a bit more)

Approach taken: We opted for a NIO based file structure (memory mapped file) to store the cells and their references. This allowed us to have a faster access to any cell in our application. We can request a given cell from the NIO storage and get a wrapper object that actually only reads the data from the storage if it is really accessed. The lifet5ime of the wrapper objects was pretty short (get the object, render the value, release the object). These measures drove the overall memory usage down (the memory mapped file is not part of the regular memory limit of java).

We also created a differential calculation engine (our formulas were finite thus we could optimize the calculations and write code that can da a full recalculation of the spreadsheet as well as a differential calculation that would affect a small set of cells).

Love to hear your thoughts about this post

h1

Make changing your Application Easy

August 8, 2008

There we are – writing applications for a vertical market – we get some specs, we have to understand what the business people want from us and then we have to implement it. After a while we’re done with the application and then the business people find out it’s not what they wanted. Where did we go wrong? Sometimes it’s the business analysts/developers that did not get it right, sometimes the vision of the business people/process owners is not clear enough. Who knows. I think the question should be ‘what can we do so there is an easy way out of that situation’? The second problem I see is that once an application is deployed in the business world the requirements change – a new marketing strategy, new laws or just another company policy may require changes. We have to go back and change a running system, retest, redeploy, etc. My question again: What can we do so there is an easy way out of that situation?

This brings me to the following observations:

  • Business users like to change stuff around
  • Business users like to modify text and marketing information (especially if it’s customer facing)
  • Small to medium changes are required over the lifetime of an application to adapt to new situations

If we do not want to engineer ourselves into a corner and end up with work that we do not want to do in the long run we have to find a solution for the above requirements. Those requirements translate into the following for me:

  1. Make sure an application can easily be deployed
  2. Give the business users a tool to change the appearance of the application
  3. Use a DSL, XML structure or properties file to drive most of the application

Some of the projects we have done in the last two years are web based applications. I found by using a CMS or WCMS system such as magnolia we can build on a solid foundation that takes care of 1). We then store all properties used for the application in the content management system  as well sattisfying 2) and we added a form renderer and task engine to the CMS in order to sattisfy 3).  We also get full access management through the CMS and since magnolia is based on jackrabbit (JSR-170 compliant content repository) all content is stored in a way that other applications can access it as well.

Things that became easy to address because of magnolia as a base framework:

  • changing any text on the site
  • moving pages, form elements tab orders, validation
  • changing the look of the site
  • error handling
  • rewriting all form pages to support ajax
  • implementing the forms in a different framework
  • allowing form handling for different browsers in different technologies
  • staging to life deployment through an activation and review process

This list is not complete – by choosing a CMS and implementing the required functionality on top of it we are able to move a lot of the work that seems to come up because of misunderstandigns between the business users and the developers into the hand of the business users (they can take action themselves) or make the change easy for the developers (it’s just a property or two to change in a UI instead of having to change it in code, recompile and redeploy).

I’d love to hear from you what other approaches you take to solve this issue.

h1

Using Queues instead of Weaving

August 5, 2008
Everybody likes to use code weaving to make software development easier – take java pojo’s and hibernate and somehow magically we get that persistence functionality weaved in that allows us to load and store data to the database layer. It makes a programmers life easier but also hides a lot of the implementation into this cloud of code that we never see.

I tend to think of method calls as messages – I then also try to think of the thread as a queue. The messages are sent to the queue and executed. This makes code weaving the process of observing the queue and triggering actions when certain events happen on the queue. The action can alter, cancel, repeat or simply pass the message through.

I like that design very much – Swing for example uses the event queue for all it’s messages (even adding a component to the UI tree is a message). One can easily design an application to use a similar pattern – for example, if you create a queue and messages that change your model (messages would be commands) instead of a simple API to your model you can then handle things like undo/redo through the queue messages.

The Cairngorm framework for Flex makes use of messaging as well in order to make the client/server communication and interaction with the client side model and the server side model easier.

I feel using queues and messages makes it easier for other programmers to understand what is going on in the application (the queue is a much easier concept to grasp than the code weaving).

h1

Helpful Java Swing Articles

August 5, 2008

I have been doing java swing development for the last few years – whenever I start a new contract job on an existing swing project I am usually amazed that people working with swing have never heard or seen these two articles on java.net

  1. closure style actions
    Make sure you are not writing tons of inner classes to handle your actions or even use a global event handler
  2. modify swing when the components are added
    This is great to style a component when it is added to the UI according to the operating system or make a change to a component without having to implement it for every look and feel that you support


These two articles can be a great starting point to make your life developing swing easier.