Hopefully the changes to WebKit can flow through to Android as well.

What disturbs me is that incredibly few people realise just how much work is involved in implementing caret and selection handling correctly, let alone most of the other infrastructure provided by contentEditable. I wound up talking to Mark Wubben who actually does have a pretty good idea – he implemented a DOM based selection system for the Xopus editor. Needless to say he had quite the war story to tell, but despite doing what sounded like a very impressive bit of work, the selection code is still very broken in a couple of key ways: accessibility and international inputs1.

Accessibility

Accessibility support was my first response to anyone suggesting using canvas as an editor – no-one had considered it. If you play around with a screen reader for a while, you start to realise just how important the selection is to accessibility. Selecting text generally causes it to be read out for example. It’s not just screen readers either, screen magnifiers will move the screen along to keep up with the caret, ensuring the user can see what they’re typing. If the caret is fake, none of these features will work and the editor becomes inaccessible.

International Inputs

Even I’d forgotten about this one, but it’s likely an even bigger problem to solve than accessibility. The array of input devices, keyboard shortcuts and other ways that people input characters is pretty amazing when you start exploring it. People in English speaking countries don’t tend to notice this because they have quite a small number of characters in use which all fit on the keyboard. Even Western European countries have the vast majority of characters required available with a direct key sequence. However there are far more complex types of inputs around. The simplest of these is the use of “dead keys” such as acute on a US or British keyboard. To type é actually takes multiple keystrokes on a UK Mac keyboard:

1. Option-e. This is a dead-key so only a placeholder appears in the output ().
2. e. Now the e-acute character actually appears and is inserted into the document content.

Not to pick on Xopus but at least in Firefox on Mac, I can’t type é at all in the editor because it doesn’t correctly support dead-keys2.

Asian languages can have even more keystrokes required to type a character and much more complex input methods that to be usable, require changes to the caret, underlining content etc. Plus the methods and UI is different on different OS’s. While users can handle some variation, they shouldn’t have to learn a whole new input method to be able to use your application3.

The Way Forward

Instead of trying to work around the limitations in contentEditable we need to be pushing browser vendors to fix it and make it usable. Where the current APIs aren’t good enough, standardise better ones – we’re dealing with multiple incompatible APIs already so the backwards compatibility problems are limited. This needs to be a focus so that we don’t wind up in a world where online editors only work for fully-abled, English speaking users. Most if not all of what’s required is already there, it just needs some love and care to remove the bugs and better handle corner-cases.

1 – There is some good discussion of the complexities involved here from the W3C canvas list as they struggle with the accessibility implications of canvas.↩

2 – and to be completely fair, EditLive! fails to paint the placeholder, though the character does type correctly. ↩

3 – So no, Google Virtual Keyboard is not a real solution, however good an idea it is for people using things like internet kiosks with foreign keyboard layouts. Besides, would you like to be stuck typing only with your mouse? ↩

The problem between Java and Firefox is that Java always sets the ‘StartHTML’ and ‘EndHTML’ fields to -1 to indicate there is no context and Firefox will refuse to paste HTML from the clipboard if StartHTML or EndHTML is set to -1. As such, if you copy HTML from Java it will be impossible to paste into Firefox. It works perfectly with Word and IE.

I’m not 100% clear on whether the clipboard data Java is generating is valid but I consider it a bug in both Java and Firefox – Java for not being strict about what it outputs and Firefox for being too strict about what it accepts. Bug reports shall be made1.

On the Java side, the problem is in sun.awt.windows.WDataTrasferer.HTMLSupport.convertToHTMLFormat which sadly is Sun-specific, private and loaded from the bootclasspath so a little difficult to work around. There is however a nasty, but effective, hack from Peter Buettner which does indeed get around the problem. I’ve chosen to avoid the need to copy/paste HTML in this particular application but the approach is worth saving a reference to in case it’s needed later.

1 – Bug 598289 with Mozilla and there’s a simple test case available. The bug report has also been filed with Oracle and is now available in the bug parade. ↩

General Primer

• Why P Tags are Your Friends
• P Tags in Word

Solutions for Specific Situations

• P Tags and Single Line Fields
• P tags and Flash
• The Email and P Myth

JavaScript editors are all fundamentally based on a technology called contentEditable which makes a part of the web page become editable by the user. While there is a lot of functionality that’s usually bundled under the “contentEditable” title, there are a few core parts which are essential for a browser to support correctly1:

• Selection – allowing the user to position a text caret in the content and select various parts of it.
• Content entry and deletion – allowing the user to type basic text and to delete selected content.

Without these two key components of contentEditable it’s impossible to create a high-quality, WYSIWYG editor with JavaScript. Despite some variations in how things work, this functionality is available on all common desktop browsers and has been for quite a few years.

The Problem

On mobile WebKit, for both iOS and Android, the basics of contentEditable support aren’t working correctly. The problem is two fold:

1. While the user can position a text caret and change the selection, the actual caret (the blinking vertical line), never renders so users don’t know where their typing will appear.
2. The on-screen keyboard doesn’t appear when the caret is placed in an editable section. On devices without a real keyboard it’s then impossible to type.

WebKit is actually surprisingly close to getting this working, with a lot of the underpinning functionality appearing to work but these two issues prevent editors from being usable2.

Why Does It Matter?

While geeks may feel at home writing HTML code by hand or using a markup language like markdown or textile, non-technical users find it significantly easier to use a WYSIWYG editor. Currently, it’s not possible to offer such an editor on mobile devices due to the lack of contentEditable support.

Even worse is that many sites are already using a WYSIWYG editor which then prevents users on a mobile devices from contributing content – even when the rest of the site works perfectly. With mobile browsers improving so rapidly, most sites now work without needing special adjustments so the lack of contentEditable support is one of the few major incompatibilities.

How Might it be Fixed?

To be honest, I don’t know and don’t have the experience with the code involved to find the source of the problem and fix it. However, with both the source to WebKit and Android being open, hopefully someone does have the experience knowledge and a bit of time to solve this.

With WebKit being the browser engine of choice on so many devices, the fix may even flow around to a bunch of mobile OS’s.

Relevant Bug Reports

Android

• 9515 – No onscreen keyboard when inside a contentEditable area
• 9513 – Unable to select text in contentEditable field
• 9512 – contentEditable field shows no caret
• For some background see also: 8253 – isContentEditable returns true even if the browser doesn’t really support it

iPhone

Apple don’t provide a public bug list, but this issue is logged as Radar #8399288. Apple do use the number of bug reports for an issue to adjust priority so if this issue is important to you, please do log an issue and cite Radar #8399288.

Others

I don’t currently have bug reports against other systems but would appreciate pointers to any relevant bugs for other systems (whether they use WebKit or not) so they can be tracked.

1 – The other major component is the execCommand function which provides built-in implementations of a number of common commands like bold, italic, etc. While it’s certainly useful to have these, they can generally be replaced by pure-JavaScript implementations without sacrificing quality or performance. ↩

2 – I’m unaware of any mobile browser that supports contentEditable well enough to make an editor feasible. In fact, WebKit appears to be the closest to that milestone. ↩

The list of HTML tags supported by flash is very short, so you will need to disable most of the editor functionality, however since flash does support things like bullet lists, for the most intuitive editing experience, it’s best to use P tags to break up the paragraphs rather than BRs.

Instead of avoiding P tags to avoid the extra white space, use a two part approach. In the editor, use P tags for paragraphs, and include the CSS:

p { margin-top: 0; margin-bottom: 0; }

Then before passing the HTML into Flash, filter it to replace <p> with <span>1. Store the original content, with the P tags, in the database so that they are still present when the user goes back to edit existing content. You will need to disable any formatting that applies to the paragraph element – most notably alignment. There is no way to support alignment without a block tag and P is the only available block tag within Flash. If you need alignment support, you will have to use P tags and put up with the extra white space. Lists will still work correctly2 because the editor is working with the P tags, and the Flash renderer

In any case for Flash you will need to disable a lot of functionality to ensure only the supported elements are inserted. Essentially, bold, italic, underline, font face, font size and font color, images and hyperlinks. There are a few other CSS properties supported, but still very limited.

The limitations in Flash do provide significant challenges to providing a powerful and intuitive HTML editor, but the user experience will almost always be better if the editor works with P tags, and then they are replaced before rendering, using CSS within the editor to ensure consistent rendering. In all cases, it is vitally important to heavily restrict the functionality that is available in the editor. If at all possible, avoiding the need to render HTML within Flash is usually the best option, but not always feasible.

1 – Flash 9 doesn’t support the DIV tag, otherwise that would be the ideal alternative. ↩

2 – tables are not supported by Flash anyway↩

<p><%= fieldContent %></p>

In that case, where the output is the only thing within the paragraph, the best solution is to leave the paragraph tags in editor content and remove them from the rendering template. That gives the editor the context it needs to work well and avoids a lot of potential problems with rendering the content. If however, you’re including some extra content in the same paragraph then the situation is more difficult, e.g. your rendering template is more like:

<p>Summary: <%= fieldContent%></p>

In this case, you need to include the P tags within the rendering template and need to ensure that any functionality in the editor which requires inserting a block tag is disabled. Many people think that just switching to insert BR on enter will achieve that, but it’s quite a long way short. For example, if the user creates a list, the template would output:

<p>Summary: <ol><li>Content</li></ol></p>

which is invalid. Similarly since alignment must be on a block tag, it would create:

<p>Summary: <div style=”text-align: center;”>Content</div></p>

which is also invalid. Headings, tables, margins, blockquotes and many other functions will cause these problems. Instead of trying to disable the problematic things, it’s much simpler to only enable the inline formatting options that you need – it should be very limited since you’re only editing a small snippet of content:

• Bold
• Italic
• Underline
• Strike-through
• Font face
• Font size
• Font color
• Background color
• Superscript
• Subscript
• Images
• Hyperlinks

Depending on your editor, there may be a couple of other functions that can work, but most things will need to be disabled. Notably, all these functions work regardless of whether there’s a P tag present or not, because they work “inline”, directly on the text.

Once you’ve restricted yourself to these functions, it’s usually ok to switch the enter key to insert a BR instead of a P, but carefully consider whether having line breaks in the content makes sense with your template anyway. If you want users to be able to insert multiple paragraphs, you should design your template to let them use P tags rather than using BR as a substitute. Consider using a plugin to disable the enter key altogether since the field is designed for a single line response.

Finally, using a plugin or validation when the user submits, check that the content has only one paragraph and isn’t too long to fit within your template. With many editors it will be better to work with the content inside a P tag within the editor and then use JavaScript or server-side filtering to remove them when saving the content.

The bottom line is that when restricting content to a single paragraph, you should ensure that all functionality which works at the paragraph level is disabled in the editor as it cannot be made to function correctly. You should also carefully consider whether users should be allowed to insert line breaks in the content and if they are, whether these are really paragraphs and users would be better served with a slight change to the rendering template, allowing them to use the full functionality of the editor.

The first line is a paragraph, note that it ends with the invisible paragraph marker character (¶). The second line ends with a line break character instead (¬) so the end of the line is not the end of the paragraph. There are actually only two paragraphs in this document. You can see this in action when we position the caret at the end of the text and apply a heading style.

The heading style is a paragraph level style, so it applies to the entire paragraph, even though there’s a line break. In fact, if you save that original document as HTML, you get the structure:

<p>Paragraph 1</p>
<p>First line of paragraph 2<br>
Second line of paragraph 2</p>

Just like any good HTML editor would create. You can try this yourself in Word, typing enter will create a new paragraph, typing shift-enter will instead insert a line break. The same keystrokes apply to most HTML editors as well.

Rendering Differences

When copying and pasting content from Word into a HTML editor, it’s possible that there will be extra space added between each paragraph. This is a result of having different settings for what Word calls paragraph spacing (called margin-top and margin-bottom in HTML/CSS).

With the more recent versions of Word (last 10 years worth on Windows, last 5 years or so on Mac) the default is to have no paragraph spacing which effectively makes a paragraph marker and a line break (P and BR) look the same, simply moving to the line immediately below1. In any case, in browsers the default is to apply a 12pt paragraph spacing through the CSS margin-top and margin-bottom attributes which are set in the browser's default stylesheet.

So to get the same rendering as Word, you should keep using paragraphs but add the CSS:

p { margin-top: 0; margin-bottom: 0; }

Make sure you add it both to the rendered page and to the styles used by the editor. There is more detail on this and other ways to make HTML behave more like Word in the Ephox article, Making EditLive! Content Behave More Like Word, which is applicable to all editors.

1 – I've never quite understood why they made this change as it has led to users inserting a new paragraph by pressing enter twice which effectively adds an empty paragraph to get the paragraph spacing they want. ↩

Take for example, a simple function like alignment. In Word and any other sensible editor, when you click the right align button, the current paragraph becomes aligned right. Which pretty clearly highlights the problem – if you don’t have paragraphs, what should become aligned? Typically what happens is the entire document, being one paragraph, gets aligned right and user’s unsurprisingly complain. So why not just change the alignment of the current line? Think about it in HTML terms, if you have:

Line 1<br>
Line 2

Where do you apply the text-align style exactly? You can’t apply text-align to inline elements so:

<span style=”text-align: right;”>Line 1</span><br>
Line 2

doesn’t work. Further, we’ve been told that inserting P tags isn’t allowed, so we can’t just make it:

<p style=”text-align: right;”>Line 1</p>
Line 2

because that would introduce “extra whitespace”1. So the only option is to insert a DIV tag which makes the HTML structure quite complex. DIVs aren’t just a meaningless element, they add structure to the document and the editor can’t tell which DIVs are important structure and which are just pretending to be paragraphs2.

This problem crops up in a huge number of different places – applying headings, style classes, lists, indenting and more. All these special cases need to be handled, adding to the download size for the editor and reducing performance – not to mention taking up a huge amount of development time that could have been put into something more useful.

WYSIWYG editors do such a good job of hiding the way that HTML and CSS work, that people often forget that in the end we’re still trying to generate high quality, standards-compliant HTML output. We’re limited by what those standards and browsers can actually do, so removing important concepts like paragraphs makes it incredibly difficult to create an editor that works intuitively.

Ultimately, it’s not that we developers don’t want to support your use case, it’s just that the restrictions you’re asking us to abide by dramatically reduce our ability to deliver a high quality authoring experience.

1 – which can and should be removed using CSS in the first place ↩

2 – remembering that what was once an unimportant DIV may become important at some point because of a change to scripts or CSS that the editor isn’t aware of ↩

p {
  margin-top: 0;
  margin-bottom: 0;
}

The other group however are people writing HTML emails who insist that P tags are completely incompatible with some email clients and cause rendering problems. At one point it seems that Yahoo! Mail really did strip them entirely but now it just applies a different default CSS styling (ironically, the magical CSS above that removes the extra spacing). So if you naively use P without specifying the padding you want, some clients will display a blank line between paragraphs, others, notably Yahoo!, will push all the lines immediately under each other. The solution is of course the opposite magical CSS incantation:

p {
  margin-top: 0;
  margin-bottom: 1em;
}

Solved right? Nope. This runs straight into the where the heck do I define styles? problem. In HTML, it should be:

<html>
<head>
<style>
p {
  margin-top: 0;
  margin-bottom: 1em;
}
</style>
</head>
<body>
…
</body>
</html>

However while this works in some clients, it has no effect in most. Instead, the common wisdom is to move the style tag into the body tag:

<html>
<head>
</head>
<body>
<style>
p {
  margin-top: 0;
  margin-bottom: 1em;
}
</style>
…
</body>
</html>

Which works almost everywhere. Enter GMail. GMail never respects the style tag, only inline styles. So now you need to write your paragraphs as:

<p style=”margin-top: 0; margin-bottom: 1em;”>…</p>

Thankfully you can use the margin shorthand if you know what you want the left and right margins to be as well:

<p style=”margin: 0 0 1em 0;”>…</p>

I would strongly recommend using embedded styles while editing and then just use post processing to inline all the styles – Premailer can do that for you.

As far as I can tell, there is no need to avoid P tags in email anymore and sampling a number of emails from various clients that happened to be in my inbox, they turned out to appear in emails from a few different clients though that’s far from scientific and it was still intermingled with a lot of <br> and <div><br></div> hacks. I would be very keen to hear from anyone who knows of an email client that cannot be made to render P tags correctly.

With a bit of luck we may be able to start moving away from the horrific abuses of <br> tags…