We are thrilled to announce that the 2019 DIAGRAM Report has been published and is live on our website. An accompanying blog post is live on the Benetech website. A truly communal effort, this year’s report features chapters written by DIAGRAM community members who are experts in their respective fields. We’d like to extend a huge thank you to Alan Harnum for authoring the chapter on Personalized Learning, Richard Ladner for writing the chapter on Accessible K-12 Computer Science, Emily Moore and Jenna Gorlewicz for their co-authored chapter on MultiModal User Interfaces, Doug Schepers for providing the chapter on Accessible Data Visualizations, and Bruce Walker for his chapter on Sonification. Their participation has made the 2019 edition of the DIAGRAM Report truly special.
We are particularly excited about the accessibility and UX responsiveness of this year’s report. Using accessible coding best practices, we have built a report that works for people using screen readers as well as those viewing the report on a mobile phone. We’ve also included enhancements to a few of the chapters, including audio sonification of graphs, an accessible SVG graph, a heat map accessible table, and a fully accessible embedded PhET simulation!
We hope you find this year’s report interesting, educational, and a valuable resource. Please feel free to share it broadly and as always, we welcome any and all feedback.
The 2019 DIAGRAM Report is well underway! For those unfamiliar with our annual report, it is a compilation of input gathered from renowned experts in the community to highlight some of the most promising technologies and trends changing the educational landscape for students with disabilities in the United States. This year’s report will focus on multimodal interactions, personalization, accessible computer science, data visualization and sonification – the use of sound to convey information found in previously inaccessible content (e.g. a bell curve).
Although we have previously written chapters on multimodal interactions and personalization, the technology is moving so fast that it is necessary to continually provide updates on the latest advancements. Additionally, we also have discussed coding in previous reports, but the DIAGRAM community felt it necessary to include an updated section on accessible computer science since it is a growing field, for which we need to ensure that persons with disabilities have the opportunity to participate.
Since charts and graphs are generally a challenge for people with disabilities to access, especially those with vision impairments, we will explore ways that data visualization and sonification can be used to access the information provided in these images.
The 2019 DIAGRAM report has an expected release date of August 31, so make sure to check back for the announcement of when it has been published.
In the final quarter of 2018, the collaborative multimodal study between DIAGRAM, Mad*Pow, and NCAM concluded. Through the study, we learned that the 25 students representing varying disabilities exhibited a preference for modalities that aligned with their individual learning preferences and personal desire for varying levels of information. These preliminary findings and additional details from the study were shared with the Tactile and Research Working Groups in the first quarter of 2019 and received with much interest. Members from the Research Working Group have expressed particular interest in further examining and analyzing the associated data and have submitted a conference proposal to the Council for Exceptional Children based on the preliminary findings. We continue to look for additional opportunities to share the information and look forward to working more closely with the Working Group members around this study.
The Mathshare project boasted several new developments this quarter. The addition of Alex Cabral, a UX Researcher to the DIAGRAM Team, enabled us to conduct more user studies, distill feedback, and better plan priorities in the coming months. We also began integration pilots with edtech platform providers, including EdReady, and are continuing to discuss collaboration possibilities with additional edtech partners to reach more students. Finally, we presented Mathshare to a virtual audience of mostly special educators in a webinar entitled, “Getting Started with Accessible Math.” Hosted by CAST, the webinar was co-presented with other leading math tool providers, including EquatIO and Desmos and took place January 21, 2019.
The Accessible Interactives Library is an online GitHub repository of best practice accessible code snippets for common online interactions, such as drag-and-drop, sliders, and photo carousels, with an online demonstration of their real-world implementations. Since the initial launch in August 2018, we’ve worked with dozens of publishers and edtech developers to update, disseminate, and track implementation stories regarding this resource. To date, our collection of code samples and examples includes 24 accessible interactives and adoptions from key partners such as W.W. Norton and PhET Interactives. These implementations were shared and met with much excitement during the “Speed Geeking” rounds, held at the Strategic Planning Meeting. We plan to continue growing the collection throughout the remainder of the DIAGRAM + award period.
The Book Industry Study Group released an updated Quick Start Guide to Accessible Publishing on January 18, 2019, accompanied by an article in Publishers Weekly to help promote awareness about its utility. Although the original Quick Start Guide was completed in 2016, this revised update covers new advancements to help publishers create more born accessible content that addresses the needs of a broader audience. Alignment of standards that enable the prospect of Born Accessible also paves the way for new features such as an EPUB 3 with proper HTML tagging, ARIA attributes, and accessibility metadata. Since its recent launch, some publishers have already been calling the publication their “go-to guide for accessible publishing.”
The DIAGRAM Standards Working Group is developing two new reference documents. The “UX Guide for Displaying Accessibility Metadata for EPUB” will help libraries and bookstores promote accessible EPUBs so that students can easily find these accessible books and ensure they will meet their needs before purchasing them, and the “Accessibility Summary Authoring Guidelines” will help publishers create meaningful accessibility summary statements to be put into EPUB metadata. The summaries will ultimately assist students, teachers, procurement offices, and others in finding the right accessible EPUBs for their needs. After gathering feedback from the publishing community, we plan to publish both of these guides next quarter.
The 2019 DIAGRAM strategic planning meeting was held on February 28 and March 1 in Leesburg, Virginia. The primary goals of the meeting were to:
- review outcomes and learnings to date in the DIAGRAM project
- share and highlight tools and resources developed by the community
- identify emerging technologies and areas where accessibility support is at risk and needs to be addressed, both immediately and longer-term
- analyze and strategize on sustainability priorities and plans for our collective efforts
- agree on priorities and next steps for moving forward, including updating charters for working groups.
60 people attended, of whom, 12 were new to the DIAGRAM community and 25 were first-time participants in a DIAGRAM Strategic Planning Meeting. Over the two days, 
participants explored topics such as what usability means for various disability audiences and ways to produce larger-scale impact (e.g., through adoptions, dissemination, or spheres of influence). They also discussed how technology shifts impact our priorities and work in the world of standards/best practices development and tools creation. And as always they continued to examine how to better work with DIAGRAM’S ever evolving and growing community, whether as individual contributors with specialized skills, adoption partners, or connectors who can help us reach and influence the broader technology and edtech market. Working groups also had opportunities to meet, which resulted in the following updates:
- The Outreach Working Group and Content Working Group combined, with plans to assemble real-world examples of how DIAGRAM tools and resources are being used by students, educators and parents, content producers, and edtech partners.
- A new working group addressing user feedback and testing was formed.
- The Developers, Standards, Tactile, and Research Working Groups had an opportunity to meet in person, recruit new members, and set concrete priorities and goals for the coming months.
All and all it was a fun and productive meeting, that was well-received by participants, with 95.6% of individuals describing the meeting as either productive or extremely productive.
On May 15-16, DIAGRAM held its 4th annual code sprint which was co-hosted with Partnership on Employment and Accessible Technology (PEAT). This year’s sprint took place at the LightHouse for the Blind and Visually Impaired in San Francisco who generously donated space in their building. Sponsored by Microsoft, without whom the hackathon couldn’t have happened, the event took place during the 16th International Web4All Conference and attracted a professionally and geographically diverse crowd.
Participants came from as close as San Francisco, Palo Alto, Sunnyvale, and Redwood City, and as far as the UK, India, Switzerland and Portugal, with backgrounds in academia, industry, government, and, nonprofits. Regardless of where they were from and what their background was, everyone came together to learn and explore the potential of new technologies to personalize the web and provide an accessible user experience for all. It was a fun, informative, educational and productive two days. One participant and longtime DIAGRAM community member Neil Soiffer summed it up well stating:
“The hackathon was a great opportunity to learn and do some good work. I knew something about accessibility, but nothing about Jupyter notebooks [one of the projects]. My hackathon partner Paul knew something about Jupyter notebooks, but nothing about accessibility. By working together, we both learned something and were able to make lots of small but significant accessibility improvements to Jupyter that we expect will land in main Jupyter code.”
The work done on Jupyter is important not just because of the exchange of new information, but because Jupyter is a new endeavor for DIAGRAM with huge potential for impact. As long-time and often-quoted community member Sina Bahram pointed out:
“Jupyter is used by millions of people around the world. It is a foundational technology in data science, finance and many more fields. By enhancing the accessibility of this environment, we are able to begin moving the needle slowly but surely towards a day where students and professionals with disabilities are not excluded from the most foundational and essential tool required to participate in school and work.”
Evan Yamanishi, DIAGRAM community member representing publisher W.W. Norton, went on to say,
“I wish I could have worked on every project at the hackathon this year and really would’ve liked to learn more about Jupyter Lab. My group focused on defining a production-ready version of the enhanced visual descriptions from 2018, hacking on previously unsolved UX issues such as keyboard dragging and both improved and reduced motion. Clayton Lewis was especially great at documenting everything and asking challenging questions about affordances and customization things that are required to gain wider adoption of enhanced descriptions.”
Day One kicked off at 8:00 am on a very rainy Wednesday. Despite the early hour and shockingly bad weather for May, the excitement in the room was palpable. Not only 
were people pumped to hack for good, but in a twist on our usual code sprint format, this year, there would be judging and prizes — computer bags and Amazon HD fire tables donated by Educational Testing Services (ETS) — for the winning teams. The room was abuzz with lively conversations, brainstorming, strategizing and the occasional breaks for cheeky antics and a ton of food.
One of the highlights of the day was a tour of the LightHouse facility. The beautiful space boasts huge windows and panoramic views of downtown San Francisco and City Hall. It was designed by Chris Downey, LightHouse board president, and realized by architect Mark Cavagnero, both of whom are blind. The space is completely accessible using different floor textures to designate the type of room a person is in. It also has walls that are lined with tactile art and tactile maps as well as conference rooms, a lab, craft room, exercise space, teaching kitchen and more. Everyone was extremely impressed with the space, both the layout and the 
history behind it, as well as the mission of LightHouse for the Blind and Visually Impaired. You can learn more about their work on the LightHouse website.
Day One ended around 9:00 pm with hackers completing their initial code and one participant, Kesavan, generously offering to help event organizers bring the leftover food to a local food bank. We couldn’t have asked for a better start to the hackathon.
On Day Two the room was much quieter with the hackers intent on writing and cleaning up code in preparation for the presentations and judging. The atmosphere in the room was tense though teams still somehow found time to goof off. They were also treated to demos by Stanford graduate students Alexa Fay Siu and Tayo Falase. Alexa, who is working toward her PhD in mechanical engineering and human-computer interaction, showed off her project, shapeShift. This process takes a 3D STL file and renders a 3D 
object using a tactile array of small cylinders that can be raised, rotated and resized in real time. This project can also be used by students with blindness, visual impairments or a learning style better supported through touch to be able to fully explore graphs and other mathematical concepts in a way they can better understand.
Tayo, who is receiving a master’s degree in engineering, human-centered design, demonstrated her project, Tactile Code Skimmer, that represents the tab indentations within computer code on a physical device that showed eight lines of where the indentations were for each line of code. The goal is to track your place in the code and identify the level of indentation on this external device by feeling where the sliders are located. As you step through your code, the sliders will move to correspond to the tab indentation for that line of code. The level of indentation is meaningful when programming. For example, an inner loop of instructions should all be at the same indentation level when the tab indexing is incorrect. If it is not, it could indicate a potential bug in the software.
The afternoon wrapped with the teams putting the finishing touches on their code and then, finally, the presentations happened and judging took place. If you are interested in the presentations, they were recorded and can be accessed on DIAGRAM’s YouTube channel. Participants waited with bated breath (aka eating street tacos while the judges deliberated). As it turns out, all teams did extremely well making significant progress over the two coding marathon days, with many producing usable code. The judges had quite the challenge on their hands and tensions ran high (low) as the teams anxiously (calmly) waited to learn how they placed.
The teams and their projects were as follows:
- JupyterLab: Over half of the participants worked on one of three JupyterLab projects (making menus & dialogs accessible [low hanging fruit], high-level architectural changes required between the main project and its libraries, and improving the overall Notebook Experience) – an interactive development environment that enables users to create and share documents that combine live code with narrative text, mathematical equations, visualizations, interactive controls, and other rich output. Four teams worked on adding various features to Jupyter Lab that allow screen readers to recognize and verbalize menus, tabs, buttons, modal dialogs, and fields.
- Accessible Code Repository (Accessible Interactives) featuring Charles LaPierre (Benetech), Markku Hakkinen (ETS) and Candida Haynes (Independent) – The DIAGRAM Center is assembling a repository of open source code for common interactions such as synchronized text-to-speech highlighting, carousels, page settings, date pickers, drag and drop, and so on. The team worked on improving the layout and functionality of the repository and plans to add more best-in-class code that can be used to make applications more accessible.
- Accessible Extended Image Descriptions featuring Evan Yamanishi (W.W. Norton), Matt Nupen (Benetech), Clayton Lewis (Boulder Colorado), and Tammy Speed (Founders and Coders) – They say a picture is worth a thousand words, but when you can’t see that picture or understand it due to a visual or cognitive disability, it’s as if the image is not there. Authors and publishers need a way to add simple image descriptions as well as more detailed descriptions that appear in an unobtrusive yet easily accessible way. The team worked on an interface with Accessible Rich Internet Applications (ARIA) web standards that allow a user to create extended descriptions that can be toggled on/off and repositioned.
- Accessibility Conformance Testing featuring Carlos Duarte from (University of Lisbon), Pawan Kumar Patel (IIT Kanpur), Marie Trudelle (Empowerment Through Integration), Damien Engels (Google), and Ramit Garg (Intuit) – the team proposed new rules for testing the accessibility of links, link text, headings, widgets, and other elements of HTML to ensure adherence to ARIA standards that allow screen readers to be able to verbalize and interact with web pages. Automating this function makes it easier for web developers to make sites accessible from the very beginning.
In the end, the judges could only pick three winners. In third place was the Accessibility Conformance Testing project. In second place was one of the Jupyter Lab’s teams that made menus and dialogs within Jupyter Lab accessible, and in first place was the 
Accessible Extended Image Descriptions team which showcased a number of options for providing extended image descriptions in an inclusive way as well as documentation to support this work. Of course, the real winners are the people who will benefit from the code produced over the two days so we would like to again extend a huge thanks to Microsoft for their sponsorship allowing us to host the hackathon and providing swag for all of the teams, ETS for providing prizes, to the LightHouse for the Blind and Visually Impaired SF for providing the space , and to all the participants who volunteered two days of their time to come up with solutions to accessibility problems. Their dedication and passion are truly inspiring.
It’s hard to believe we are already at the end of another year and what a whirlwind year it’s been — full of burgeoning prototypes, web page launches, new partnerships, and many other exciting updates. As we head into 2019, we’d like to leave you with some highlights from our collective endeavors in 2018. Thank you to our amazing DIAGRAM community for contributing your time towards making all this work possible. We look forward to an even more exciting 2019!
- The second annual DIAGRAM Report provided a look ahead at technology for students with disabilities and their parents and educators. This year we had three guest writers from our community write two of the five chapters; a big thank you again to Clayton Lewis, Lisa Dieker, and Amanda Lannan for sharing their expertise. In case you missed it, be sure to read about Accessible Coding, Artificial Intelligence, Augmented and Virtual Reality, Multimodal Interactions, and Personalized Learning in the 2018 DIAGRAM Report.
- We made significant strides in the development and testing of our new product, Benetech Mathshare, an interactive math tool intended to help students better express their work while performing online math problems.
- This year’s third annual DIAGRAM Code Sprint was co-sponsored by and hosted at the Microsoft Conference Center in Sunnyvale, California. This was the first year the sprint spanned two days and was a roaring success with more than 50 expert participants from all over the world.
- Benetech Imageshare was revamped and we teamed up with the Tactile Working Group to assemble and create various multimodal resources. These resources were tested with end users, and DIAGRAM also engaged with an independent user experience firm, Mad*Pow, to expand the testing methodology they first worked on as part of the Accessible PEEP and the Big Wide World project in partnership with WGBH’s NCAM.
- We had an amazing six months collaborating with our resident advisor, Clayton Lewis. Clayton’s generosity and willingness to share his decades of work and expertise in research and accessibility, particularly in the area of cognitive disabilities, was invaluable. We look forward to continuing to work closely with our dear friend, who has since returned to home base at the University of Colorado, Boulder… and thank you.
- The digital standard for EPUB and EPUB accessibility got some significant attention this past year. With a new EPUB 3.2 specification soon to be released in 2019 and the launch of Benetech’s Global Certified Accessible program ensuring publisher EPUB books are accessible.
This fall, DIAGRAM worked with Mad*Pow, a strategic design consultancy, to better understand the impact of using multimodal image resources to understand complex mathematical and scientific concepts. Mad*Pow conducted 25 one-on-one interviews with students who were diagnosed with one or more of the following: low vision, intellectual disabilities, autism, and learning disabilities. The session was divided into two topic areas: frog anatomy and the Pythagorean Theorem, and each participant was presented with an image with text description, 2D tactile graphic, and 3D manipulative model for each topic. Half of the participants were presented with the Pythagorean Theorem first and half were presented with frog anatomy first.
Overall, Mad*Pow found that participants exhibited a preference for modalities that enhanced their understanding of the subject matter by clarifying the relationships between elements, in this case, the relationships between digestive organs in a frog and the variables in the Pythagorean Theorem. 3D was the most popular modality for frog anatomy because its interactivity helped participants gain a deeper understanding of organ connectivity and the digestive pathway. 2D was the most popular modality for the Pythagorean Theorem because it demonstrated the relationships between variables in an easily understandable and visual way. And per our original hypotheses, participant preferences also seemed to reinforce individual learning styles and proclivity toward varying amounts of information (e.g., more or less detail). We look forward to sharing the outcomes of this informal study in the coming months and have also begun a similar study to learn more about the usability of Mathshare. For more details about the findings of the tactile or math studies please contact lisaw[at]Benetech[dot] org.
In 2018, DIAGRAM launched the newly revamped Benetech Imageshare, an open source platform that enables educators and consumers to find and share multimodal resources related to key science, technology, engineering, and mathematics (STEM) concepts. In addition to adding new metadata to facilitate search and discovery, the collection also allows educators and end users to add instructional teaching notes, production notes, and ratings and comments after using specific resources.
We look forward to adding more accessible resources and content into the collection in 2019, including feedback currently being collected by the Tactile Working Group about the usability of specific resources in the collection. Read more about the testers in the Tactile Working Group update.
2018 was a huge year for one of DIAGRAM’s newest tools, Benetech Mathshare. Not only were we fortunate to secure funding from three additional organizations to continue the work through 2019, but the tool continued to receive positive support as we showcased the prototype to educators and students.
Version two of Mathshare, which debuted this past fall, included enhancements to the visual design, workflows to support the creation of custom problem sets, and the ability for students to save and share their work. Since the launch, we have been working on classroom pilots to collect feedback about these new features, including the usability of end-to-end workflows for educators. In the final weeks of 2018, we have also been engaging in larger-scale pilot discussions with integration partners and wrapping up dozens of in-depth student feedback sessions, including formal studies through our partners at Mad*Pow. We look forward to an exciting 2019 for Mathshare and continue to invite educators interested in piloting the new version of Mathshare in their classrooms. Contact sueannm[at]Benetech[dot]org to get involved.
In 2017, we launched the inaugural DIAGRAM report that explored six technologies predicted to impact schools now and in the future, particularly for students with special needs. We followed with the second annual DIAGRAM Report in August 2018 that contained updates on Personalized Learning, Accessible Coding, and Multimodal. We were honored to have some distinguished guest writers: Clayton Lewis wrote a chapter on Artificial Intelligence while Lisa Dieker and Amanda Lannan contributed a chapter on Augmented/Virtual Reality.
As soon as the report was published, we began working on the 2019 report. If you have not already had an opportunity to weigh in, we ask you to share your insights about what we should focus on for the next report by filling out a brief survey: https://www.surveygizmo.com/s3/4590803/DIAGRAM-Report-2019 (please complete by January 4, 2019). If you are interested in sharing your expertise or being a guest writer, please contact Charles LaPierre at CharlesL[at]Benetech[dot]org or Lisa Wadors at LisaW[at]Benetech[dot]org.
With 2018 wrapping up, we are actively planning for lots of exciting developments and gatherings in 2019 and beyond. One of the major events scheduled for the coming year will be the 2019 DIAGRAM Strategic Planning meeting, which will be a two-day affair in the Washington, D.C. area. This will be another opportunity for the DIAGRAM community to strategize on the vision and future of the DIAGRAM Center. In addition to sharing goals for the remaining 18 months of the current DIAGRAM + award, we will also be thinking big about the types of impact that our community can bring to advance accessibility and inclusive education. If you are a DIAGRAM advisor or an active working group member, but have not yet received an invitation to this event, please contact Amaya[at]Benetech[dot]org. Outcomes of the strategic planning meeting will also be shared in the next DIAGRAM newsletter, so stay tuned!