What happens when you turn AAC platforms into instant messaging devices? These 2008 Tech in the Works winners found unexpected discoveries such as the critical need for chat specific vocabularies.

2008 NCTI Technology in the Works Award Winner — Point-and-Chat Software

Principal Investigators:

« Benjamin Slotznick, JD/Ph.D. President, Point-and-Read, Inc. » Jeff Higginbotham, Ph.D. Professor and Director of the Signature Center on Excellence in Augmented Communication, SUNY – Buffalo

Download the Point-and-Chat final report (PDF).

Building A Little Suspense: When Things Go Awry

When you are engaged in a creative technical enterprise some things don’t work out, and sometimes they’re fairly important to the project!

With this straight-forward admission, Jeff Higginbotham, an expert in social interaction, augmentative communication and technology design, opens our case study involving the first Instant Messaging (IM) system for youth with severe communication difficulties. As he foreshadows, some of the potential inherent challenges that can occur — despite excellent project planning — form part of the central lesson that other technology collaborators can learn.

Far more importantly, this case study underscores the critical value of collegiality and relationship even in a short-term collaborative endeavor. Jeff, who served as the academic researcher on the team, continues his cautionary statement, adding:

You could see that for people who weren’t flexible, this could have been a real test to the undertaking. In fact, any one of the parties could have accused the other one of not holding up their end, and you would have, right there, the makings of the undoing of the project!

IM Makes Sense for AAC Users

Back to the beginning. Ben Slotznick is the visionary designer with the original “gut instinct” that something like Point-and-Chat was needed for AAC users, especially those who cannot read or write. He joins Jeff in pointing out that in many respects, Instant Messaging is becoming more important as a communication venue of choice to youth than face-to-face conversations or the ‘old technology’ of telephone interaction. Relying on Pew Internet research statistics, they point out that IM is used even more than email by adolescents. In their grant proposal, they argued that this emerging emphasis on text-based messaging had the potential danger to further exclude youth with communication disorders from natural human communications that are so critical to their ongoing development.

However, they recognized that the movement toward IM also presented an opportunity: a specifically tailored system could actually put these kids on a much more equal communicative basis. This could happen in part because of the message formulation lag time that is accepted and expected as a normal aspect of use of IM, which occupies a space on a sliding continuum between synchronous and asynchronous communication.

The fundamental concept of Point-and-Chat, then, is to combine customized input icons and screens of existing AAC devices with screen reading technology and the first IM interface specifically designed for these users to send messages, ensuring relative ease and familiarity for youth with communication disabilities.

Jeff describes the potential impact eloquently:

The basic foundation for literacy and for any communication is the ability to carry out a conversational interaction. It’s a fundamental building block of our culture. [IM] is one way to help an individual enact those kinds of important communication performances in the Internet world in a more symmetrical, parsimonious relationship. This can lead not only to a more independent life as people communicate with friends, family, and teachers, but a full life.

Connections Lead to Collaboration

Prior to the project, Point-and-Chat was substantially developed owing to eight years of Ben’s work. He had also already been applying his screen reader technology with another collaborator, Dave Hershberger, President of Saltillo Corporation. Now, he realized Point-and-Chat was ready for a new stage in the development process. As he explains,

You need to collaborate with different people for different aspects of life and work. When I am doing start up development of software — essentially ‘garage development’ — I’m working with what and who is available locally in a very iterative process to develop the technology. To actually test it, I need to work with someone very different, who instead of being immersed in this technology for years and years, can look at it with fresh eyes and can try it in fresh situations, with new and different or somewhat different populations. I needed to work with someone like Jeff.

Saltillo already had a working relationship with Jeff, and made the initial suggestion for a connection between them. Ben travelled to hear Jeff speak at the 2008 CSUN Conference, with the leverage of a specific hope. He had attended an NCTI presentation a year earlier that introduced him to its Technology in the Works competition, and had this opportunity in mind as the most appropriate mechanism to establish the process for, and quicken the pace of, a long-distance collaboration.

As they conversed after the session, Jeff saw the potential for groundbreaking research related to his own university work. He also owns a small private consulting and development firm, and decided it would be best to undertake some aspects of the work that would be burdensome with university processes through his more agile, private company.

Instant Messaging for AAC – Maybe Not So Instant

In their proposal (PDF), Ben and Jeff stated their intent to, “Evaluate the performance and usability of the Point-and-Chat software by documenting the learning and use of the technology by 5 different individuals who use direct selection AAC systems… analyzing both structured and spontaneous use of the program.”

There were, however, unexpected challenges in finding suitable and willing research subjects. Moreover, the project quickly “morphed” into the necessary execution of far more extensive development process than initially conceived. Ben elaborates:

When we started the research, Point-and-Chat required users to work at a computer using a mouse to select buttons or areas on the screen to make it work. Now, we had to allow stand-alone AAC devices to be able to control the computer by having the computer take a keystroke signal from them. It required much more extensive coding than we expected to add this keystroke control for every command.

Long-Distance Collaboration

Jeff, who has also been involved in technology development with other technology vendors, responds first to Ben’s description of unfolding events graciously, adding, “I want to point out that all of this is part of any research and development activity.”  However, he is direct about the impact, continuing,

We have only met face-to-face once, so this had definite implications for our ’starting-out’ relationship. Ben’s work began to expand and we had to change our priorities in the face of some very significant challenges before we could go back out and use the technology with people.

Ben explains, though, how they proceeded with tenacity as they saw the need to know how the product might work with differing populations who might be in the market,

For us this came at a crucial time when we’re not yet big or established enough to get a large- scale grant, but could test some things out. That required us to really look hard at how it was working in the field with different people. Just because you are thinking of a group doesn’t mean that it is that population who will buy a product. We wanted innovative ways of dealing with changes in design for potential users, and believe that can enhance the experience for everyone.

Tech in the Works as Researcher-Vendor “Speed-Dating”

As the collaborators converse about the process, they interweave the nature of NCTI’s work to form collaborations and relationships, and specifically the Technology in the Works award itself, as fundamentally connected to the experience. They both laughingly agree with NCTI’s explanatory term for the award as a form of sponsored ’speed dating.’  It expedites both the processes of meeting people with common interests and the establishment of effective collaborative projects. Ben indicates the selection of an award that didn’t involve “reams of pages in order to write a proposal” and a flexible funding agency was as important as the discovery of a cooperative and capable working partner in case “the research didn’t go totally according to plan.”

In this vein, Jeff describes “taking a 10,000 foot view of this,” and seeing that the nature of the NCTI award was still perfect in many respects for the position Point-and-Chat was in. For one thing, it lacked the degree of “high-stakes” risk, cost, and impact of an SBIR or NIH project. With access to reports on past NCTI-funded projects, Ben had also noted that,

Other projects also had bumps in the road. I knew we had to work not just between ourselves, but to develop a relationship with this supporting agency that would allow this to happen and not penalize us in the end. NCTI’s support allowed us to develop the relationship we needed — to maximize the ‘getting-to-know-you’ aspects while minimizing negative consequences.

Jeff indicates the equanimity and people skills they both brought to the project were crucial:

Our work has occurred over the telephone. I cannot speak for what Ben does during his day, but it’s cacophony on my side with classes, department concerns, emergencies, and students. I’ve got other projects taking my time, so being able to present a calm demeanor in the storm we’re all in is really important. When we’re dealing with the problems of the task, we’re not dealing with interpersonal problems.

Commercialization of Point-and-Chat

To end any possible remaining suspense, Point-and-Chat has reached the point of commercialization as a more informed and powerful product than it was before the project. It has benefitted from “incorporating something like 95% of the recommendations from users and this research,” according to Ben.  It is available in Saltillo’s ‘e-shop‘ and printed catalog, and a 30-day free trial is available.

Running on a computer, Point-and-Chat can be controlled with nearby AAC devices such as DynaVox Series 4 and 5 units. Full keystroke control is now built in, and operation by switch-scanning is substantially developed in preparation for the next version.

Users typically select a photo icon to choose their communication partner, and graphical icons in an AAC program or system, such as Saltillo’s DesktopChat™,  to generate a message that dynamically appears converted to text in a box in Point-and-Chat. This permits users to check and reformulate messages if desired. Incoming messages arrive in a separate text box, and can immediately be read aloud with a variety of simple selection options (short messages are vocalized automatically). The software runs on desktops or laptops, and can be used in conjunction with the Saltillo Chatbox 40XT unit. Testing on a handheld computer version is expected to begin this summer.

Ever the academic researcher, Jeff’s view is that Point-and-Chat “is fairly far down the road, and opens up the ability to be doing other kinds of collaboration, working in other areas of communication technology.”  Ben concurs, adding, “I said I needed to work with someone like Jeff. I’m glad it was Jeff, and I would like to work together more. Hopefully we can find a project!”

Unexpected Discovery: IM Specific Vocabularies are Still Needed

In addition to powerful new coding and abilities that were added in large part based on the feedback of research subjects and their families, there were additional successful outcomes for the collaboration. Perhaps the most important of these was the first known research discovery that specialized and predictive ‘chat’ vocabularies for this population need to be developed that will assist in streamlining IM conversation and making it more comprehensible. Ben and Jeff feel linguistic analysis will be necessary to determine the best type of IM-specific chat vocabularies that should be developed separate from those vocabularies that already exist in full-scale AAC devices.

Lessons for Effective Collaborations

As signaled earlier, the first half of the central lesson of this collaboration is that Things Go Wrong, even for experienced and capable developers and researchers. The ability to maintain flexibility in conception, thought, and action is often essential in developing new technology.

The second component they demonstrate so clearly is that Being Calm and Collegial Moves Project Forward. This simple form of human capital — the discipline to put people first — remains at the core of successful collaborations, and Ben and Jeff believe this is true even as the stakes climb higher.

Beyond this valuable advice, Ben and Jeff suggest that project partners should:

1. Specifically evaluate partners for maturity and calm demeanor. These collaborators agree that the relationship they created, the obstacles they overcame, and the growth they experienced were personally as gratifying as the successful outcomes of the technology itself. Going straight to the point, Jeff sums up by saying, “It’s so dependent on the maturity and the emotionality of the people involved.
2. Run a ‘cost/benefit’ analysis. Jeff suggests,

   “When you’re a busy person, you have to make decisions about who you’re going to work with, how much energy it’s going to take for what you’re going to get out of it… I don’t just mean instrumentally — the commitment that ‘we’re going to finish the project together’ — but at what cost emotionally and in every other way.”

3. Understand that technology development and collaboration are both complex. These collaborators recommend keeping the incredible number of variables and complexities in mind when unexpected situations arise — this can help to diffuse the potential for tension between parties. Ben adds, though, that you should take steps to “visualize what could go wrong or what you’ll need next.”
4. Account for differences between business and academic worlds. Jeff begins,

   Universities and businesses perform two different functions around innovation. Universities are typically set up not to produce a product, but to focus on process. We spend time trying to figure out and deeply understand how something works, but without the same level of urgency, know-how, or horizon-point that a manufacturer has. Our completion schedules are out of kilter with the manufacturer who is trying to feed families of his employees, and their tolerance for software bugs is much lower than a university prototype. The cultures are different, but to go back to this NCTI grant, the competition provides the different groups opportunities to work together, to try out things that are instrumental for long-term relationships.

   Ben agrees there are differences should be understood, and focuses primarily on the relative intellectual and collegial isolation he has experienced as a developer with an idea that will be patented. He feels the structured opportunity to gain interaction with, and input from, a university team is profoundly beneficial to break an idea out beyond one person’s conceptions and blinders about “what is simple to use and what isn’t.”

   Notably, Jeff also says the experience is benefitting not only Ben’s company, but his own, adding: “I’m going to be bit more ‘reality-based’ in how we need to operate and what our flexibility will need to be going forward.”

5. Stay with the commitment and “Don’t freak out about it!”. This is Ben’s simple, summative advice.  Jeff agrees, adding,

   “There may be 500,000 reasons why a piece didn’t work out, and the only important thing is to fix it. (Rather than engaging in blame), you’ve got a much more important goal of moving things forward!”