Symphonious

$ unzip ~ephox/vmware-debian-etch-r0-mini.zip
bash: unzip: command not found

Might have been just a little bit picky when I first installed that box….

So we've got a new internal system that we've built on top of JCR. Currently we're using Jackrabbit as the repository, but eventually it will be ported over to something like IBM Portal or something like that. Unfortunately, right now we're deploying the app to a pretty limited server - both in terms of CPU and RAM.

It turns out that using Jackrabbit with the Derby persistence manager in that kind of situation is a horrible, horrible idea. Everything works great on systems with modest amounts of CPU and RAM but once we deploy to that poor little virtual server in the sky page load times skyrocket and the whole thing becomes unusable.

Profiling showed a few things we could fix but didn't solve the problem, so we set out to do some testing with different persistence managers - we already have MySQL running, why not try that? Well, mostly because the JCR export produces XML and sadly with Jackrabbit (and possibly with any compliant repository - not yet sure) that produces an invalid XML file which can't then be imported. Turns out putting binary data in XML files doesn't really work - particularly if that data includes characters like  and � which aren't valid in XML even if you entity encode them. At this point I could just reimport the data again from scratch, but since I know we're going to need to migrate repositories again in the future I need a reliable way to export and reimport, not to mention the fact that it would be nice to be able to back up and a stable format instead of the random binary formats that form Jackrabbit's native configuration.

Sigh.

Ken Coar complains about some of the changes in FireFox 2.0 and mentions:

My basic plaint is as usual: when changing the user interface, don't violate the Principle of Least Astonishment and force the change on the user. Make it the default, perhaps, but always provide a preference option that lets the user keep the old UI behaviour. The user should be in charge of changing his work habits, not the software.

I couldn't possibly disagree more. The rule is if you're going to force the user to change their behavior - make sure it benefits them. In other words, don't make changes that make your interface less efficient and less easy to use. Preferences are not something that make people more efficient - the preference dialog in almost every application is confusing for users and generally unfriendly, the more stuff you put there the worse it gets. You need to keep the number of preferences to a minimum - focussing only on those areas where user needs are truly different - and make your UI encourage pr enforce the most productive work habits (while balancing intuitiveness).

If you make a bad change to your UI that's a problem, but adding a preference to revert it isn't a solution, it's a second problem you've introduced.

If ever there was a problem that just wouldn't die it has to be inter-project dependency management. If your code depends on external libraries, it's pretty simple to pick whichever solution you prefer - either grabbing a version from a repository or checking jars into your source control. However, if you depend on a project that you control, it gets so much messier.

If the projects are small, it's probably a good idea to just set up the build system to build them all together - effectively making them separate projects even if for development purposes you can just build this bit or that bit and utilize precompiled versions of the dependencies.

Once the projects get to a certain size though it simply takes too long to go recompiling everything unless you really need to. So you want to set up a repository for your internal projects that you can just grab builds from. That's great until you're out on the road and want to compile - suddenly that repository isn't so accessible. The other problem is you always want to be able to reproduce the build with the exact same contents - including the exact build of your dependencies. This implies that you have the particular version number used checked into source control and that you can get that build again - even if your repository goes belly-up (worst case you have to rebuild the dependencies from source control too).

Of course, if you have a particular build number checked into source control, you're not building against the very latest so if things break you won't know about it until much later when you upgrade. This is where Ephox is at and it's just not good enough. If you always build against the very latest though, it's hard to tie things down when you're testing for a final release.

There's a couple of things we can do to improve the situation:

1. When you depend on some behavior or API in another project, write a test for it in that project and give it an obvious comment describing which project needs that behavior. Then if the behavior or API changes, the project just won't build and someone will have to go and fix it.
2. I think we need to set up a nightly build that takes the Gump approach. It should build every single project, always using the very latest version of it's dependencies, then if the build passes, it should automatically commit the version identifier for the dependencies so that when the developers come in the next day they'll all develop against that build (which they can pull from the repository). This neatly avoids the development delays of having to checkout and build dependencies, while still keeping things as up to date as possible and providing fairly rapid feedback if things break.

That leaves just the problem of doing offline builds - something that's not been a major problem for us before but on a recent project has been a key requirement. The solution we have for that project, which seems to work quite well, is to have the build pull in the dependencies and cache them where they aren't deleted when you do a build clean. Then, simply provide a property that can be set to tell the build to work completely offline and just use the cached version. There's still a few glitches with it but generally it's working really well.

So if someone wants a project while I'm away on my honeymoon, come talk to me this week and I'll give you an overview of how I think we can make it work.

That permanent generation had better provide some pretty damn amazing optimizations or I'm going to add it's inventor to my list of people to track down and torture at next JavaOne. It turns out you can only reload a Tomcat webapp so many times before the PermGen space fills up and everything dies. That's annoying enough in development but it means I can't upload a new WAR file to our test server regularly without also bouncing Tomcat. Right now our continuous integration server (Bob the Builder) is doing just that after every commit so the server isn't staying up all that long.

Mock objects for unit tests will also run afoul of PermGen space and apparently something in Eclipse does too because every so often it dies on me too. Sigh.