Blog Posts Business Management

Millions of sign-language users can’t communicate with your organization

Blog: Capgemini CTO Blog

Many sign language users do not know any spoken languages, especially if they have been deaf since birth – they can’t speak them and they can’t write them. In many cases, giving a sign-language user a piece of paper and a pen is about as helpful as asking me to communicate using Chinese pictograms.

It’s a common mistake to assume that a sign language is just a way of communicating in French or English or some other language using hand gestures instead of sounds or written characters. In fact, sign languages are entirely different to spoken languages. A person able to communicate in French Sign Language, for example, does not automatically also know how to communicate in spoken or written French.

The sound of one hand clapping

There are no reliable statistics for the number of sign-language users globally, but extrapolating from the number of people born deaf suggests hundreds of millions of people use a sign language as their first and often only language.

Organizations can overcome spoken language barriers by providing written translations or automated text translation, but there is no such shortcut for sign languages. Right now, organizations that routinely provide multi-language services have no way of providing the same convenience for sign-language users.

We are not the first to notice this deficiency. Many attempts have been made over the years to automatically translate sign languages into spoken languages using gloves that sense sign-language users’ finger movements.

The problem with this approach is that it only works with a specialized part of sign languages called manual alphabets. Manual alphabets are used to spell out words from spoken languages in much that same way that anyone who knows the Latin alphabet can spell out a word in any language that uses that alphabet, even if they don’t speak it.

Just because a sign language user can spell a word, doesn’t mean he or she understands it, in the same way that an English speaker can spell out a word in Spanish without knowing what it means.

The bigger picture

Sign languages commonly include hand, arm, head, shoulder, torso, and facial movements as part of their articulation. A sign language translation system that only looks at hand movements would be like a spoken language translation that only translates nouns – you might get a broad sense of the communication, but you would miss a lot of the meaning.

This is why visual recognition systems are now being used in sign language interpretation. Properly trained AI-based software can identify the hand and body movements of a sign-language and translate them into a spoken language. We have built a solution that does exactly this, enabling more people to communicate seamlessly in retail businesses, schools, hospitals  and a wide range of other public places.

The solution consists of in-store kiosks equipped with video cameras and screens. The recognition platform, based on TensorFlow and PoseNet technologies, interprets the user’s signs to text or audio for a remote or in-store customer service agent. Their reply can be in signs, text, or audio.

Sign up

It’s important to note that translating from a sign language to a spoken language is just as difficult as translating from one spoken language to another, if not more so. We cannot claim to have built a solution that will allow sign-language users to communicate flawlessly with non-sign-language users, but we have built a solution that makes a good level of communication possible in a situation where otherwise it is often impossible.

Any business that recognizes the competitive importance of giving their customers the best experience possible should also recognize that this applies to all their customers, not just the ones who know spoken languages. If your business is committed to inclusivity, get in touch with me and we can talk about building a hassle-free experience for every one of your customers.

Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples