Symphonious

There is an interesting tendency in software development to try to keep the internal model as closely in sync with the output format as possible. When you control both, that's probably not a bad idea - it simplifies serialization. What is important is that the user interface matches up precisely with the user's mental model of how to use your software. Attempting to keep your model simultaneously close to the user's model and the output format often forms contradictory goals.

I had a recent experience that really highlighted the power of knowing the difference between the user's model and the output format: determining when to use a space and when to use a non-breaking space to allow users to have adjacent white space in HTML. The HTML standard makes white space insignificant so two spaces together render as if there was only one space. This is very useful for making your HTML readable, but it doesn't match up with the user's expectations when they use a WYSIWYG HTML editor. When a user presses space twice, they want two spaces - ignoring the second press of the spacebar would be frustrating.

Our original attempt at meeting this user expectation was to insert a non-breaking space if the user pressed space with the caret after an existing space character. This worked pretty well, except that when the user typed a single space at the start of a paragraph, that space was lost. So we added code for that case too. Then the user deleted a word, leaving two spaces together. Then they typed text between two spaces, removing the need for the second space to be a non-breaking space. Everywhere we looked there was a new case that we had to handle. All these special cases are hard to keep track of and our internal builds started becoming more and more confused about when to insert a non-breaking space1. It drove us nuts trying to use our internal wiki and constantly discovering that the new page we were creating had a non-breaking space entity in the title because we confused this poor, overly complicated code for managing white space.

It turns out the solution is very simple. We'd been trying to keep the internal model in sync with the output format, instead of with the user's mental model. In the user's mind, white space is important and two adjacent spaces mean two adjacent spaces. In fact, even our internal model was happy with that because the Swing text APIs work with a variety of formats and render two adjacent spaces as two spaces. The only problem was that when we serialized, those adjacent spaces suddenly became insignificant. So we changed the serialization code. Now, when you hit space twice, internally we get two adjacent spaces. When we serialize the model though, we detect those two spaces and change the second one to a non-breaking space3.

1 - You can see this in the source code for many of my older posts2, since this blog runs the latest internal build as its post editor.↩

2 - and you can see it in some recent posts too because somewhere along the line a client requested that shift-space should insert a non-breaking space and I keep accidentally hitting shift-space instead of just space. It's going to become a configuration option soon.↩

3 - the parser will make sure that when loading in a document, the white space is interpreted according to the normal HTML rules, and we still add extra white space when serializing to make the HTML source readable, it's just the user's actual data that gets non-breaking spaces as needed.↩

A while back I saw a comment go by that raised my interest, it was something along the lines of: we know all our users want a better editor, but it's just not something we can charge more for. My recent experience with Yojimbo and what I've seen of a number of people evaluating blogging systems, content management systems and anything else content-centric, tends to indicate that even if you can't charge more, you'll definitely sell more.

If Yojimbo had an awesome editor for it's notes, I would have bought it at twice the price, even without any of the other functionality being up to scratch for what I wanted. It would be so useful to have a central location to store notes, with an awesome editor to quickly edit them that I'm thinking of writing such a system myself. A home-grown solution would lack all the polish that Yojimbo has, but I'd use an off the shelf, top of the line editor so 99% of the time I used the app I'd be getting a smooth, polished experience.

Similarly, blogging systems live and die by whether or not they provide a good experience for writing posts. You can make most blogging systems provide whatever output you want to the end user, it's the admin side of it that users see most. What part of the admin system do they spend the most time in? The post editor.

Wikis are even more editor-centric. Again, getting the output to be readable and searchable is a pretty simple task that pretty much every wiki does pretty well at - making it smooth and intuitive to edit pages is where wikis originally shine and where there is a lot of potential for improvement. Providing a plain text field and wiki markup just isn't good enough anymore.

More and more, people are judging mail clients based on their editor too. Sure, I still prefer plain text email but the rest of the world is falling in love with rich formatting and you'd better bet they want their mail clients to provide top notch editors for crafting their messages. Apple cottoned on to this with their ready-made templates in Mail for OS X 10.5. I'm going to hate that feature with a passion but you bet it will be hugely popular.

The editor is where the user spends all their time, so don't underestimate the amount of power it has in a purchasing decision. The number of places where editor technology is becoming crucial to the user experience is growing every day and the more you hear about user generated content, the more important editors are becoming.

I've been playing with Yojimbo for the past week or so and it shows a lot of promise but just doesn't quite make it across the line to something that would be useful. Essentially Yojimbo allows you to store stuff in a central location. You can add text files, web archives, PDF files, bookmarks, serial numbers and passwords. You can categorize them, label them and most importantly search them. It sounds promising and I'm sure that at some point it will be pretty awesome, but overall I found it disappointing. There were a few limitations that make it far less useful for me than it should be.

Firstly, the notes are almost plain text - it has about the level of rich text support as TextEdit - no tables, no lists and worst of all, no easy access to anything. All of my notes wound up being plain text just because it was too difficult to apply formatting. The biggest missing feature here is lists - 99% of what I want to put into Yojimbo are lists of things and it doesn't give me an easy way to do it.

The web archives were really cool, a quick AppleScript and I could create a web archive of the page I'm viewing in Camino. There are just two problems - they only display in the built-in panel or Safari and I couldn't find a way to get the original URL back out of it. It's rather difficult to blog about something interesting if you can't quickly and easily get the original URL back. Only opening in Safari isn't a major issue, but it's annoying that they didn't respect my preferred browser setting.

Bookmarks are just that, with no extra functionality over what a browser has built in. They do open in your default browser correctly but since they don't have a web archive attached they still aren't great for searching.

PDF support is similar, it works and there's nothing special about it. The major disappointing factor for general files is that there's no way to edit them in an external editor and have the changes reflected in Yojimbo. The whole application is one great big data silo. I would have liked to keep track of some play scripts I'm working on in Final Draft with Yojimbo but every time I edit it I have to delete the old version from Yojimbo and readd a new one. I have very few documents that never change so Yojimbo's file support is wasted.

The most disapointing part of Yojimbo though was it's exceptionally primitive AppleScript library. Given that it's an application that is meant to be a central hub of information, I can't believe how primitive the AppleScript API is. Essentially, you can create a new item of whatever type and in theory retrieve them again, though I just got an internal script error whenever I tried that. My AppleScript is pretty primitive so I've probably just messed up the script. Even so, where is the API to set which collections an item is in? How about searching through the database quickly and easily?

Finally, the syncing requires .Mac. Now, I know that's because it uses SyncServices and Apple in their infinite wisdom have decided that should only work with .Mac accounts, but it's lousy none the less and it's BareBone's decision to use a crippled Sync foundation, so it's them that gets the blame for the lack of openness. Even if the non .Mac syncing was more primitive, some attempt should at least be made to avoid the need for an expensive .Mac account.

Overall, Yojimbo is the app that should have revolutionized the way I manage my digital odds and ends, but all the features it has are just the bare minimum to tick the box without providing enough functionality to actually be useful. BareBones seem to have gone for delivering a large number of features in the shortest possible time and wound up with a product that can do lots of stuff, but is good at none of it. I'll be going back to good old file system folders and dashboard sticky notes.

Slashdot linked off to an old interview with Kent Beck and Cynthia Andres, but it was interesting to see the number of people complaining about the pair programming aspect of XP. Comments like:

Programmers are solo beasts - putting two of these dragons behind one keyboard is asking for trouble.

and:

In the OO Programming course I had this year, they encouraged us to practice extreme programming. Well, it sucked… Everytime my partner and I encountered a little problem and one of us had an idea, he had to explain it to the other, which took at least 3 times as long as just typing it out, which in almost every case made the idea perfectly clear. Suffice to say, after about 4 hours we gave up and just sent each other any files we had edited (of course making sure we didn't work on the same file at the same time).

show a major lack of understanding, not only of pair programming, but of the real world in general. Firstly, there's no reason when pair programming that you shouldn't just type out your idea instead of spending excessive amounts of time waving your hands around and trying to explain it. Code talks and tests talk even better, so use them as a method of explaining. Secondly, and most importantly, if members of your team can't effectively work together and communicate clearly, you have a major problem with your team that will reduce your chances of success in any methodology. In XP such a scenario almost eliminates any chance of success.

It's not just your chances of success that drop when you have a poorly gelled team, it's also your teams morale and work satisfaction. XP should be fun to do. If it's not, you either aren't doing it right or you have a bad team. At Ephox, we tend to spend a lot of our day laughing and generally being happy. That atmosphere comes both from having a gelled team, but also from a number of XP practices that encourage social interaction.

The most significant practice in this regard is sitting together. Being locked away in your own office means that you have to make a special effort to interact with your work mates. Not only do you lose the serendipitous communication of overhearing important stuff, but you also lose the social interaction, the personal touches, that turn work mates into friends. Doesn't that mean that you waste time getting interrupted by talking about what happened on the weekend? Sure does. Does it mean you enjoy yourself at work more, have better developer retention rates and ultimately go faster because happy workers are productive workers? You bet.

Your pointy haired boss might want you to have your nose to the grindstone constantly, but that's because it's notoriously difficult to measure actual productivity so instead pointy haired bosses use number of hours on task as the measure. If they can make you work longer hours and never be distracted from your task then obviously you'll be more productive right? For some things like mind-numbing production line work that is probably true. For work where you need to your brain to be sharp and fresh, not a chance. When you need to think clearly, you need to give you brain a rest regularly and not push yourself to hard. When you do work, it should be energized work.

If you want to be successful with pair programming, you need to get on well with your work mates. You don't have to be the best of friends, but you need to be able to sit and chat, laugh about things and be comfortable around each other. Pairing with a team full of prima donnas and antisocial types would be a nightmare1 so it's vital that you focus on building a gelled team, not just a team who can solve silly math puzzles.

You might think that this is a requirement just of XP, but it's not. It's a requirement of enjoying your work. Even pointy haired bosses know that enjoying your work is important, why do you think they spend so much on company christmas parties and the like? What they don't realize is that one-off events are an insignificant impact on morale compared with having teams that enjoy working together all the time. When done well, those one-off events are designed to get the team to come together, get to know each other and gel better.

At Ephox we consider a gelled team so much a necessity that we make the last step in our interview process going out with the entire engineering team for coffee. There's no technical questions, just sitting around chatting with the people you'll work with. Earlier parts of the interview process will tell us if you have the right skills, but they only give us a vague idea of whether or not you'll fit into the team2. Going out with the whole team though, lets us make sure that there's no major personality clashes with anyone in the team, and that you fit in with the team culture, spirit and people.

So what do you do if you're in a bad team? Either get rid of the antisocial types, or get out yourself and find a team that knows how to work together and have fun. Feel free to send me a résumé.

1 - actually, spending any time with prima donnas and antisocial types is generally unpleasant↩

2 - as a hint, your interview at Ephox has gone well if you and the interviewers have laughed through most of it. As a corollary, if you spend most of the interview laughing and the interviewers don't, you probably didn't do so well. It's bad in any interview if the interviewers spend the whole time laughing and you don't - but we're too professional to do that. ↩

If you happen to run a lot of JUnit tests in one go with Eclipse, you may find that towards the end they start to slow down or throw OutOfMemoryExceptions. They do that because the Eclipse JUnit plugin creates a new instance of the TestCase for each test method, calls setup, test method, tearDown and then for some reason keeps the object around. This means that if your tearDown doesn't null out all the fields in the test class you can wind up with a whole lot of wasted memory.

You'll have to have a whole heap of atomic tests before you notice this, but with about 1200 integration tests, today we started getting out of memory exceptions. Fortunately, there were only a few base classes that we needed to add clean up code to and everything's running smoothly again.

It's pretty annoying that Eclipse is wasting memory like that - why should we waste developer time just because the JUnit plugin is clingy?