A post from Shady Characters

Emoji, part 9: going beyond

This is the most recent in a series of eleven posts on Emoji (😂). Start at PART 1 or view ALL POSTS in the series.

Given all we’ve seen so far in this series, it becomes natural to wonder: what’s next for emoji? And how do we even begin to answer that question?

We saw in part 7 that emoji are neither a language nor a script. But if we might be permitted for a moment to call them script-like, then, of all of the scripts and script-like things that we use to communicate online, emoji were perhaps the first to be native to the digital world. They were born to inject life into Japan’s teen-friendly poke beru, or pagers; later, they were adopted by Apple, Google, and other companies who have made their money online; and, under the care of the Unicode Consortium, they continue to be tended to by a group of nerds of the highest order. (As a software engineer by trade, I say that with the greatest respect.)

As such, it should come as no surprise that emoji have been, and continue to be, darlings of the tech industry. Emojli, the emoji-only social network, may have folded back in 2015,1 but that same year saw online payment company WorldPay muse that emoji might reasonably replace numbers when it came to PINs, reasoning that a combination of four distinct emoji makes for a significantly more secure password than four distinct digits.2* Also in 2015, Snapchat, an edgy messaging service popular with younger users, added emoji to indicate relationships beween users; a year later, Facebook, a distinctly non-edgy social network, augmented its internet-ancient “like” button (👍) with a palette of five additional “reaction” emoji (❤️, 😆, 😮, 😢 and 😠).4,5

Apple, as befits one of emoji’s earliest adopters in the West, have worked emoji especially hard. In 2017, the newly-launched iPhone X came with what Apple called “animoji” — animated, three-dimensional emoji with the ability to replicate the user’s facial expressions.6 It sounds odd, and, well, it was; within a day of the iPhone X’s unveiling, Devin Coldewey of TechCrunch opined that “Animoji are dumb and I detest them”.7 Dumb or not, Apple have since doubled down on the “weird animated emoji” front, last year launching a “memoji” feature that creates custom, emoji-style stickers based on the user’s facial appearance.8

A selection of 3D "animoji"
A selection of “animoji”. (Image courtesy of Apple Support.)

In this way, emoji continue to shape how we interact not just with each other but also with our computers, tablets and smartphones — and they’re starting to physically shape those devices, too. Since 2017, Apple’s high-end MacBook Pro laptops have incorporated a narrow touchscreen that displays an illuminated palette of emoji in compatible applications.9 And in October 2019, Microsoft went one better by releasing a brace of physical keyboards that allow the user to display an emoji palette with the tap of a dedicated emoji key.10,11

A keyboard containing an "emoji" key
Microsoft’s Ergonomic Keyboard with emoji key, announced in October 2019. (Image courtesy of Microsoft Deutschland.)

For all this joie d’emoji, it bears mentioning that, like the tech industry itself, emoji have never been exactly democratic. They are expressive, yes, but restrictive at the same time, constraining users to the standard set of emoji agreed by the Unicode Consortium and no more besides. Nor is that the worst of it, because the companies that control the software that runs on our devices sometimes fail to let us use even those standard symbols. From 2016 to 2019, for example, Samsung devices did not display the Latin cross (✝️) or the star and crescent (☪️).12 These omissions had mundane technical explanations, but it is not difficult to imagine more sinister motives for suppressing such culturally significant symbols. For a more egregious example, look no further than the protests against Chinese rule that currently rack Hong Kong. Back in 2017, Apple modified its iOS software at China’s behest so that devices sold on mainland China would not display the Taiwanese flag emoji (🇹🇼); at the time of writing, ‘🇹🇼’ has disappeared from onscreen keyboards in Hong Kong, too.13

In most ways that count, then, emoji are a classic walled garden of the kind for which the internet is infamous — only rather than our favourite music being corralled into Spotify or YouTube Music, or our messages restricted to friends who happen to use Facebook or iMessage, it is our ideas that are moulded to fit the emoji that we’ve been given. Just as Johannes Gutenberg’s printing press hammered spelling and punctuation into conformity by sheer multiplicative force, so emoji act as a kind of straitjacket for language, smoothing out what we want to say by restricting what we can say.

Of course, emoji are not static, and, as we’ve seen previously, copious new entries arrive on a yearly basis. If you can write a persuasive enough proposal, and you can stand the generally sedate pace at which it will percolate through the Unicode Consortium’s committees and message boards, you may be fortunate enough to see your own new emoji appear in the pages of the Unicode standard. But this does not make emoji open. Before you can type a new emoji on your phone (or even view it there, for that matter), one of Google, Apple, Facebook, Microsoft, Samsung or their ilk must decide to add it to their keyboards or apps. Then, to transmit it to your friends, their favoured vendors must do the same, lest they see instead the sad, anonymous form of the Unicode REPLACEMENT CHARACTER (�).14 For the moment at least, these tech giants wield far more power than you or I in deciding what emoji can say right now, and where they are going in future.

But there is change in the air.

To a casual observer (or, hell, a moderately obsessive observer such as your correspondent), the Unicode Consortium does a good impression of never having been entirely comfortable in its role as emoji arbiter. It is very much a case of Unicode having had greatness thrust upon it: emoji’s current world-spanning profile is a far cry from the consortium’s earliest involvement, in which it quietly and unfussily helped Google rationalise Japan’s competing emoji sets. The journey from there to here has not been without a certain amount of self-reflection.

In 2014, as Unicode considered which new symbols it should adopt in the first big emoji update since 2010, Mark Davis and Peter Edberg, respectively Unicode’s president and technical director, wrote a lengthy document explaining why it was necessary to add new emoji. They acknowledged, even at that early stage, that emoji’s presumed future as a walled garden — and Unicode’s role in tending to it — were not sustainable:

Unicode will never be able to encode all of the wide variety of emoji-like images that people seem to want to use in message, documents, and social media. The longer-term solution is to facilitate including these as embedded graphics (aka “stickers”).15

More philosophical concerns emerged as emoji grew in stature. Profiled at Buzzfeed in 2016, Jennifer 8. Lee (once, an emoji insurgent who agitated for the adoption of a dumpling emoji; now a vice-chair of the Unicode Emoji Subcommittee) described a typical meeting of Unicode’s emoji mandarins:

When new proposals for the next set of emojis came up on the docket, members discussed them methodically, debating every nuance of each picture or character. Egg was passed easily. Milk was trickier: How should it be represented? To use a carton was too American for a global language. Would it just be a white liquid in a glass? And then came beans. That discussion was a contentious but civil one; it concluded with no clear answer. Beans would not pass.

“By the end, they were getting a bit worked up and you could see that they understood this was kind of a ridiculous thing to be doing,” Lee said of the meeting. “It was like, ‘We’re engineers — we shouldn’t be debating the semiotics of beans!’” 16

Alternatively, consider typographer Michael Everson’s response to a 2017 proposal for a “frowning poo” emoji:

[This] should embarrass absolutely everyone who votes yes on such an excrescence. Will we have a CRYING PILE OF POO next? PILE OF POO WITH TONGUE STICKING OUT? PILE OF POO WITH QUESTION MARKS FOR EYES? PILE OF POO WITH KARAOKE MIC? Will we have to encode a neutral FACELESS PILE OF POO?17

But even before Everson raged against the frowning poo, a motley band of emojineers had come together to offer a solution. Not to the presence or absence of frowning poos or other emoji of questionable taste, but to the Unicode Consortium’s need to care about their existence in the first place.

In 2016, Jennifer 8. Lee, an IBM engineer named Steven Loomis and Keith Winstein, a Harvard computer science professor, submitted to Unicode a proposal called “Coded Hashes of Arbitrary Images”, or CHAI.18 Theirs was a spin on Unicode’s existing formula for emoji that would permit users to transmit any image of their choosing as part of a plain old text document — an email; a Facebook post; a WhatsApp message — and give the recipient a fighting chance of being able to see it as the sender intended. No longer would users be confined to the paltry thousands of officially-sanctioned emoji.

Now, emoji, as they stand, rely on three things. (We’re about to get very slightly technical, so let’s number those things for ease of reference.) 1️⃣ First, each individual emoji must have a globally-unique, agreed name. 2️⃣ Second, the devices that exchange emoji have to agree, at least to some extent, on what those emoji should look like. 3️⃣ Lastly, a failure at any point in the process of sending an emoji should not be unduly disruptive. You send me a ‘😂’ and I see a ‘😂’, and even if my smartphone doesn’t know how to display ‘😂’, the worst that will happen is that I’ll see a ‘�’ instead.

When, instead, you send me a picture of your cat — well, that is not something that fits well with plain text. Even before we consider problems 1️⃣, 2️⃣ or 3️⃣, there is the issue that a single image can take up more digital storage space than an entire novel. Embedding an image directly into the text that makes up an SMS message or a tweet just isn’t practical. This means that we need to decouple the name of the image, which should be short and easily embedded in a text message, from the image data itself, which will be comparatively large and better off being transmitted separately.

First, then, how do we solve problem 1️⃣ — how do we identify the image to be sent? I may have given a photograph on my smartphone a helpful title, such as “CAT PICTURE #1754”, but your smartphone has no idea whether this is a valid name or what to do with it. We might reasonably consider using a web address instead (https://my-website.com/cat-picture-1754.jpg, for instance), so that we simultaneously give the image a unique name and tell the recipient how to retrieve it, but that isn’t quite enough — the owner of that web address could change the contents of the image, or even take it down entirely.

CHAI’s authors took a run at problem 1️⃣ by defining a method capable of assigning a unique, computer-generated name to every image that could conceivably be sent as an emoji. To do this, the image itself — that is, the pixels of which it is comprised — would be processed by a so-called hashing algorithm to produce a string of characters, or “hash”, that was unique to that collection of pixels. Where a standard emoji might be called “CAT FACE”, for example, my photograph of a cat might be given the computer-generated title of “87a2fd7684f0a2ad2a5e01”. Hashing algorithms are common in the computing world, and they are carefully designed so that hashing a given input set, such as the pixels of an image, always yields the same result. Moreover, duplicate hashes are very, very uncommon: the likelihood of two different images generating the same hash is vanishingly small.

Now, parts 2️⃣ and 3️⃣. Once an image had been given a unique name, that name could then be encoded using a set of CHAI-specific Unicode characters and sent over the internet in lieu of the image itself. A recipient that understood those CHAI-specific characters would fetch and display the corresponding image, hashing it a second time to make sure it had not changed; a recipient that had not yet been imbued with the requisite CHAI-magic would instead render a fallback emoji in their place. 2️⃣ and 3️⃣, taken care of.

The only problem was that Lee, Loomis and Winstein deliberately declined to address part 2️⃣. They did not explain how CHAI-encoded images should be sent, stored, or located — or, for that matter, who should be responsible for doing any or all of those things. As they put it,

It is not the goal of this document to address all aspects of embedded graphics, and we intend to leave to domain experts and implementers the decision on how to handle these questions.18

Nor was this the only hole in the CHAI proposal. Markus Scherer, a software engineer at Google and prolific Unicode contributor, wondered what would happen if a user was to send very wide or very tall images at odds with emoji’s square icons.19,20 Mark Davis, Unicode’s president, saw that a CHAI image could even be used as a security exploit: send a unique image to one target person, wait until their device retrieves the image, identify their IP address and then proceed to attack their computer.21

In the end, CHAI was rejected in May 2016 with a curt note that the Emoji Subcommittee “does not endorse this proposal.”22 For now, CHAI’s promise of unlimited, custom emoji remains a pipe dream.

And so, as much as it would like to get out of the emoji game, Unicode still finds itself at the heart of it. The semiotics of beans, so to speak, continue to make demands on the consortium’s time in the shape of proposal after proposal for new emoji.

In January this year, Mark Davis himself put forward a idea that, while it lacked CHAI’s vaulting ambition, stood to get new emoji into users’ hands with less effort all round.23 As things stand, in order for a new symbol to be considered for adoption into Unicode, it has to be demonstrated to the Emoji Subcommittee that there is a demand for the proposed emoji, and that the emoji’s level of use, were it to be made official, would place it at least halfway up the emoji-usage charts. Of course, this is a catch-22: there’s no reliable way to gauge a new emoji’s popularity without putting it in front of users, and there’s no way of putting it in front of users without getting it approved by the Emoji Subcommittee.

Davis’s proposal, called “QID Emoji”, put forward the radical idea that emoji need not be approved before they could be put on users’ keyboards. As with CHAI, things may get a little technical here, but we’ll use our emoji triad to guide us along the way. Remember, we need 1️⃣ every emoji to have a unique, globally-agreed name; 2️⃣ a way for devices to discover what that emoji should look like; and 3️⃣ a fallback mechanism to support older devices.

To solve 1️⃣, Davis looked to the Wikidata project.24 This is an effort to build a structured database describing the people, places, concepts and things mentioned on Wikipedia, and, in doing so, to assign each one of them a unique identifier called a “QID”. Douglas Adams, for example, is “Q42”;25 the blue whale is “Q42196”.26 In Wikidata’s QIDs, Davis saw a ready-made list of emoji names that could describe almost any concept one might want.

To support emoji represented by QIDs, Unicode would define a set of special alphanumeric characters called “tags”.§ Embedded in a piece of text, a string of tags corresponding to a QID would tell the recipient what object or concept should be displayed to the end user. Each string of QID tags would be preceded by a standard emoji as a fallback, thus addressing 3️⃣.

As for 2️⃣, Davis suggested that users themselves should take responsibility for displaying their favourite custom emoji. An addendum to Davis’s proposal gave the following example:

The New Zealand Kennel Club puts together a set of emoji for dog breeds, using QID emoji. It creates a web font for those emoji, and makes it freely available. On PCs or other systems that allow downloadable fonts, users can see and use the emoji.

The Verband für das Deutsche Hundewesen (the German Kennel Club) then decides to support those emoji as well; the meaning of each one of them is already established. It creates an online tool for selecting the dog breed emoji, and also produces an app for iPhone and Android with a bundled font and emoji palette.

At some point, mobile phone vendors add the ability to allow users to add emoji to their emoji keyboard. That is, people can copy a emoji (including a QID emoji), and paste it into their favorites’ palette. A user interested in dog breeds installs the New Zealand Kennel Club font onto their phone. Later, they get a text message with the QID emoji for Shetland Sheepdog (Q39058) and add it to their keyboard. They can then pick that emoji just like any of the stock ones.27

Thus, a successful QID emoji would be taken up on a broader and broader scale, allowing its proponents to point to real, live usage data demonstrating its popularity. At that point, the Unicode Consortium could rubber-stamp the emoji by inducting it straight into the standard set.

In theory, letting users create and distribute custom emoji sounds like an excellent way to reduce Unicode’s emoji burden, but whether a success could be made of it in practice is less clear. Almost a year after Mark Davis first published his proposal, many technical and usability issues remain unaddressed, and dark clouds are gathering over QID emoji in the hinterlands of the Unicode mailing lists. In November 2019, for example, Charlotte Buff, a regular and thoughtful contributor to Unicode’s behind-the-scenes standardisation work, comprehensively eviscerated the QID proposal. Colour fonts, she noted, which are needed to support QID emoji, are stranded in a minefield of competing “standards”. Mobile devices, she said, where emoji are most commonly used, often don’t allow users to install their own fonts in the first place, and nor do they provide any easy way to insert a custom emoji. Perhaps most damning, Buff wondered aloud if anyone outside the Unicode Consortium really cared about custom emoji:

[Unicode] spends a lot of time and effort developing emoji mechanisms that could in theory be used to significantly expand the emoji repertoire beyond the [standard] list […] never to be spoken of again afterwards. They are inaccessible to the common people and unattractive to major vendors. I see no reason why the eventual fate of QID emoji should be any different.28

QID emoji are not dead, not yet, but the prognosis is not good.

To come back to our original question, then, what’s next for emoji? The answer is simple: more of the same. More proposed emoji, more furrowed brows on the Emoji Subcommittee, more emoji seasons as Unicode’s annual updates reveals their bounty of anointed symbols. For all that Unicode has tried to open up the process of adding new emoji, or to allow users to bypass that process entirely, nothing has quite stuck.

Vendors, too, the companies and organisations whose job it is to send and display emoji, seem reluctant to shake up the status quo. As Facebook’s emissary to Unicode, Roozbeh Pournader, explained in a tweet earlier this year,

Apparently the Unicode Consortium is where the vendors talk with each other most efficiently with the least friction. And they are not even trying to make a separate sticker standard. They seem to like the part of the text architecture emoji lives in.29

Even if, in the face of technical hurdles and vendor ambivalence, emoji succeeded in escaping their garden walls, would that necessarily be a good thing? Ian Bogost of The Atlantic thought not, writing in February this year that as emoji acquires new, ever-more specific icons it loses some of its flexibility and useful ambiguity. No more can one ask merely, “Fancy a 🍸?” Now, one has to choose between ‘🍸’, ‘🍺’, ‘🍷’, ‘🥃’ or ‘🍹’.30

Ultimately, it may be that emoji are so resistant to being opened up to the masses because the current system just works. Emoji are universal and interoperable precisely because they are limited in number, and the presence of human gatekeepers in the loop — overworked gatekeepers who have wrangled over the semiotics of beans one time too often, but gatekeepers nonetheless — ensures that there will be no unruly explosion of non-standard emoji to spoil things.

At least, not yet.

Erin Griffith, “Emojli Emoji Social Network to Shut down”, Fortune, June–2015. 
“The Future of Authentication: The Emoji”, Worldpay, July–2015. 
Lydia Kraus et al., “On the Use of Emojis in Mobile Authentication”, in IFIP Advances in Information and Communication Technology, 2017, 265-80. 
Caitlin Dewey, “Snapchat’s Controversial Emoji Update: An Explainer for the Old and/Or Confused”, The Washington Post, April–2015. 
Amit Chowdry, “Facebook Emoji ’Reactions’: Are There Ulterior Motives?”, Forbes, February–2016. 
“The Future Is Here: IPhone X”, Apple Newsroom, September–2017. 
Devin Coldewey, “Animoji Are Dumb and I Detest Them”, TechCrunch, September–2017. 
“Apple Previews IOS 12”, Apple Newsroom, June–2018. 
Samuel Gibb, “Apple Launches New MacBook Pro Laptop With Touch Bar for Instant Emoji”, The Guardian, October–2016. 
Tom Warren, “Microsoft Has Dedicated Office and Emoji Keys on Its New Keyboards”, The Verge, October–2019. 
Irene Nadler, “Neues PC Zubehör Ab Dem 22. Oktober Auf Dem Deutschen Markt verfügbar”, News Center Microsoft, October–2019. 
Jeremy Burge, “Samsung Puts Japan Back on the Map”, Emojipedia Blog, 2017. 
Matthew Da Silva, “Apple Removes Taiwan Flag Emoji in Hong Kong, Macau in IOS 13.1.1”, Quartz, October–2019. 
Jeremy Burge, “Turn A Question Mark Box � Into An Emoji”, Emojipedia Blog, December–2016. 
Peter Edberg and Mark Davis, “L2/14-172R: Proposed Enhancements for Emoji Characters: Background”, 2014. 
Charlie Wurzel, “One Woman’s Bizarre, Delightful Quest To Change Emojis Forever”, BuzzFeed News, 2016. 
Debbie Anderson, “L2/17-393: Feedback from WG2 Email Discussion List on PDAM 2.2”, 2017. 
Steven Loomis R, Keith Winstein, and Jennifer Lee 8, “Coded Hashes of Arbitrary Images”, 2016. 
“Unicode Directors, Officers, and Staff”, Unicode.Org
Markus Scherer, “L2/16-379: Feedback on Coded Hashes of Arbitrary Images”, Unicode Mail List Archive, 2016. 
Mark Davis, “L2/18-203: Coded Hashes of Arbitrary Images (L2/16-105)”, 2018. 
Mark Davis, Peter Edberg, and Emoji Subcommitte, “L2/16-130: Emoji Subcommittee Report, May 2016”, 2016. 
Mark Davis, “L2/19-082: QID Emoji Proposal”, 2019. 
“Douglas Adams - Wikidata”
“Blue Whale - Wikidata”
“UTS #51 Addendum: Unicode Emoji QID”
Charlotte Buff and David Corbett, “Accumulated Feedback on PRI #408”, Unicode.Org, 2019. 
Roozbeh Pournader, “Apparently the Unicode Consortium Is Where the Vendors Talk With Each Other Most Efficiently With the Least Friction. And They Are Not Even Trying to Make a Separate Sticker Standard. They Seem to Like the Part of the Text Architecture Emoji Lives In.”, Twitter, May–2019. 
Ian Bogost, “How New Emoji Are Changing the Pictorial Language”, The Atlantic, February–2019. 
A study conducted in 2017 showed that in certain circumstances, emoji were indeed a practical alternative to PINs.3 
It’s worth mentioning that Unicode is not alone in managing the emoji standardisation process. As we saw in parts 6a, 6b and 6c of this series, it works in partnership with the ISO/IEC to ratify new emoji as a global standard. That said, Unicode takes the lead in accepting, debating and approving or rejecting newly-proposed emoji. 
This is not always true, but it remains the goal. 
We saw another proposed use for emoji tags back in part 6c of this series

4 comments on “Emoji, part 9: going beyond

  1. Comment posted by John Cowan on

    I think it’s worth mentioning that no character makes it into Unicode without approval not only from the Unicode Consortium but also the ISO committee. Any country can send representatives to it, and about 14 do on a regular basis. And they are always desperate for volunteers to attend meetings. This slows the process down still further, it’s true, but it does ensure some degree of democratic (as opposed to anarchic) control.

    1. Comment posted by Keith Houston on

      Hi John — thanks for the clarification! You reminded me once before about the role of the ISO/IEC 10646 committee in the emoji standardisation process, and it’s a slip on my part to have omitted them here. I’ll update this post accordingly.

      Thanks for the comment!

  2. Comment posted by Nicolas on

    I love the idea of Wikidata-based emoji, never heard it before! The only thing that it makes me afraid of is how much it might cause vandalism on Wikidata and a lot of work for volunteers.

    Otherwise, I’m often thinking that the emoji standard could be moved out of the Uni­code Con­sor­tium. The Uni­code Con­sor­tium would simply reserve a large range of Unicode for emoji use, and delegate all the work of allocating what goes or not on this range to a separate, independent entity, an Emoji Con­sor­tium of some sort. The Emoji Con­sor­tium can have different ways of working and different criteria than Unicode, something that fits emoji better. It would remove a lot of ‘silly’ work from the Uni­code Con­sor­tium that would be able to use their focus and resources on actual languages, alphabets, and scripts, and have emoji evolving in a much more fitted environment :)

    1. Comment posted by Keith Houston on

      Hi Nicolas — I think the Unicode Consortium would, as you suggest, prefer to delegate emoji work to some other entity. I’m not sure a fixed (if large) range of reserved characters is the right way to go — who’s to say how many emoji is too many emoji? — but some sort of tag-based approach would permit an open-ended number of emoji while still allowing for standardisation of the most important symbols.

      Thanks for the comment!

Leave a comment

Required fields are marked *. Your email address will not be published. If you prefer to contact me privately, please see the Contact page.

Leave a blank line for a new paragraph. You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>. Learn how your com­ment data is pro­cessed.