During these tumultuous times of COVID-19 and shelter-in-place orders, there has been a dramatic move to an increasingly virtual life. People across the nation and around the world need more community support and engagement, as well as resources to assist them through new modes of operating. With the strong community that we have built throughout the tenure of the DIAGRAM and DIAGRAM+ awards, we are now well positioned to help both students with disabilities and the broader population navigate the challenges of moving day-to-day life to a virtual setting. This quarter we dove into action, dedicating our time to planning the best way to accommodate these new and unexpected student needs and ways to roll out resources as quickly as possible during this unprecedented situation.
- The expansion of Mathshare features to support remote learning and the addition of new accessibility features to increase usability for students with a variety of learning styles.
- The creation of Imageshare 3.0, a completely born accessible site with an improved user interface and a more intuitive layout.
- The 2020 DIAGRAM Code Sprint, which was conducted in a one hundred percent virtual format.
- Participation in organizing and presenting at the “Students with Additional Needs in Remote Learning Environments Unconference”.
For those of you wondering why we are continuing to release and plan for upcoming releases of products when the official end date of the DIAGRAM + award was slated for August 31, 2020, good news! OSEP has approved a no-cost extension for us to continue our work.
Our final report of the DIAGRAM + award is live—and it’s our most exciting issue yet! This time we covered mixed reality, the internet of things, multimodal STEM documents, and accessible gaming. Each section provides an overview of the technology as well as relevant opportunities and challenges associated with using it, additional resources, and next steps for those interested in acquiring more information.
Enormous thanks to the DIAGRAM Center General Advisory Committee, Technical Advisory Subcommittee, and the Working Groups for providing important insights and resources. Special thanks go to the following individuals: Clayton Lewis (DIAGRAM community member and Co-director for Technology, Coleman Institute for Cognitive Disabilities) and his colleague, Sidney D’Mello (Associate Professor of the Institute of Cognitive Science) for their chapter on the Internet of Things; Devin Boyle (Senior Consultant at Wheelhouse Group, Emerging Technology Lead at the Partnership on Employment & Accessible Technology (PEAT) and Adviser at XR Access) and her colleague Kai Frazier (Founder and CEO of Kai XR and a former history teacher) for their chapter on Mixed Realities; Mario Konecki (Assistant Professor and Advisor to the Rector, University of Zagreb) for his chapter on Accessible Gaming; and Volker Sorge (University of Birmingham, UK and Progressive Accessibility Solutions, Ltd.) for his chapter summarizing how technology for inclusive education has changed over the span of the DIAGRAM + award.
As always, we encourage community discussion and welcome your comments on this report.
Get to Know Imageshare 3.0
Imageshare is an open source platform that enables educators and consumers to find and share multimodal resources related to key STEM concepts.
Last quarter the focus was on planning for the next iteration of the resource. Planning was based on feedback from users and an accessibility audit done on both the original and most recent versions. This quarter, Imageshare 3.0 was born. Built to be completely accessible from the ground up, it is the definition of a “Born Accessible” product and is both screen reader and keyboard accessible. The interface is also fully responsive and can be used on cellphones, tablets, laptops, and desktop computers.
The collection consists of captioned and audio described videos, textured tactile graphic files, braille files, 3D printable files, raised line files, and image description files, covering over 40 different STEM-related topics. We will work with additional DIAGRAM community members and organizations to further increase the content during the last part of Q3.
A redesign of the user interface has vastly improved the search capability of the site. Users can now search by keywords, subject areas, file types, accommodation types, and file sources. This makes the site much more intuitive, as it accommodates the many different ways people might engage in a search.
User testing began at the end of August, the results of which will be included in our Q4 Newsletter along with the official Imageshare launch announcement and a link so you will be able to check it out yourself!
Math Made Multi-Modal
Mathshare is the first of its kind, an inclusive digital math editor that gives all students (with and without disabilities) the unprecedented ability to interact with math problems online and to easily demonstrate their math skills in a digital environment. As an increasing number of classrooms transition to online learning environments, millions of students face challenges in solving math problems online and showing their work. Mathshare enables all students to learn how to solve math equations by themselves, make errors, start again independently, and show their work to educators in a digital, accessible, and user-friendly environment.
The focus of Q2 and Q3 was to expand the intended audience of Mathshare by supporting the specific needs of students with learning disabilities, while also creating opportunities for parents to also use the tool to support their learners in response to COVID-19.
With many students engaged in virtual learning, parents have become integral to supporting their child’s education. To increase at-home usage, we added an ‘other’ user role for family members that are helping their learners, as well as allowing students to add in their own problems. We simplified existing features to increase student engagement with Mathshare. Students are able to see which sets they have already completed in the dashboard, making saving and sharing sets with their teachers easier (and helpful for cognitive accessibility). We also added “next/previous” buttons to facilitate navigation, as well as specific updates to increase screen reader accessibility. For teachers, we added a community problem set library and the option to add steps to the problems to guide and support students.
We have already started to work on text-to-speech (TTS) functionality for students who need problems read aloud. In Q4 we will be conducting additional user testing for mobile enhancement. User testing conducted during Q3 returned highly positive results. Highlights of the feedback include:
- “It’s wonderful to finally have a platform that offers a true electronic math template with the ability to have rational included. Thanks Mathshare!”
- “Appreciate making accommodations without it being known to other students”
With Mathshare…“Students didn’t need me physically there, giving them the sense of independence”
- Kelly Vick, Elementary SPED teacher
- “People that need to do math for school I highly recommend @Mathshare it is a great tool and is really screen reader accessible. I tried may update with my friend @btman16 and we both love it. It’s far better than doing LaTeX and learning math concepts at the same time.”
- Taylor Arndt, computer science student who is blind
Code Sprinting in COVID
Due to COVID-19 we were unable to have the 2020 code sprint in person at the Educational Testing Services (ETS) campus in Princeton, New Jersey as originally planned. Instead, the decision was made to host it remotely.
During the second week of June, participants from across the United States—and as far away as the United Kingdom—convened virtually for three days, leveraging Zoom, Slack, and Github to coordinate work in teams on seven different projects.
The winning project was Rendering Line Charts on a Braille Display. In just three days the team, consisting of project lead Doug Scheppers of Fizz Studio and Jason White of ETS, managed to turn a rough idea jotted down in an email into a working prototype. Their novel approach enabled quick access to data visualization trends and overviews, including a Web framework that takes line-chart data and outputs a simplified line chart to a refreshable single-line braille display. It also allows a user to select a section and hear the text values. Doug and Jason were so pleased with the results that they have continued to work on the open source project, which they have named SparkBraille.
The additional projects were as follows:
Group one, led by DIAGRAM Community Manager Amaya Webster, focused on automating the Tactile Graphic Decision Tree. The group took the original tree, incorporated the expanded branches created by DIAGRAM’s Tactile Working Group, and created a fully screen reader accessible prototype of an interactive decision tree, designed to help educators determine whether the images they are using need accompanying image descriptions, tactile graphics, or 3d models in order to be most effective and accessible. Despite a few bugs that still need to be worked out, the interactive tree is expected to go live on the DIAGRAM website by the end of Q3.
The second group was led by DIAGRAM’s Technical Lead, Charles LaPierre, and worked together for just the morning of the first day. During that time the group was able to improve the Accessible Interactive’s Library by adding a much needed Web Accessibility section, including the top three tools used for checking web page accessibility and a resource from the W3C for a complete list of WCAG compliance checkers available. The group also added accessibility improvements to the site, which included the clean-up of some overlapping text. By the end of the day, accessibility audits resulted in zero WCAG-AA error warnings. On top of all that, the library now boasts a grand total of 50 items in the repository that are open source, and freely available, software components and tools which can be used to develop accessible, interactive Web applications.
Group three was also led by Charles LaPierre. The group helped with the cleanup effort of the DCMP (Described and Captioned Media Program) video database. Together they have organized the metadata for 400 videos, categorizing each by subject and adding keywords to enhance search capabilities.
Evan Yamanishi of W.W. Norton led the fourth group, which built on last year’s project on Extended Image Descriptions to put together a more generalized solution for progressive enhancement of image descriptions, whether they are set with aria-describedby, aria-details, or even longdesc. In addition to last year’s overlay experience, the group also sketched out a modal dialog experience in which the user can expand an image to reveal more details in a universally designed way.
The fifth group was led by Benetech’s Mathshare Product Manager, Alex Cabral. The group combined their efforts on the accessible sketchpad project to build on the existing Mathshare palette. The accessible sketchpad was designed for students who benefit from thinking through a problem visually but can’t draw it out, including those with dysgraphia or other fine motor disabilities. The accessible sketchpad allows the creation of shapes that can be manipulated—changing the radius of a circle, for example. This feature has yet to be incorporated into Mathshare, but it is our intent to do so in the future.
The sixth group was led by Neil Soiffer of Talking Cat Software. This group focused on a Web proposal for putting SSML in a web page so an audio rendering of the page can be improved. During the sprint, they created a test web page that inserts a webpage (either local or from the web) and allows a person to mark up portions of it with SSML. The page also included TEXTHelp’s SpeechStream buttons.
Students with Additional Needs in Remote Learning Environments Unconference
DIAGRAM, in partnership with IES, OSEP, Bridge Multimedia and CAST, put on an unconference titled “Students with Additional Needs in Remote Learning Environments”. The unconference targeted parents, educators, and students and, with a total of 3,000 registrants, was a terrific success.
On April 8th, 2020 we hosted our second Publisher Faceoff virtually as part of the new DAISY webinar series. The 78 attendees represented 45 different publishers, many of which were in the education space. Each publisher submitted an EPUB file for an accessibility audit. The files were then compared to each other, with accessibility best practices highlighted and shared between participants. Showcasing the strides other publishers are already making towards accessibility and specific ways they can do better encourages participants to stay competitive and improve their processes, resulting in an increase in the overall accessibility of EPUBs in the publishing industry.
Standards Working Group
The group spent Q2 and Q3 working on the accessible math in EPUB test book and the advanced image description book, along with user testing for both initiatives. As the testing concludes, DAISY will be writing a report on current best practices on math, chemistry, and extended image descriptions. They will also provide a forecast of what to look for in the future based on the two test books and work done by the accessible chemistry task force.
Tactile Working Group
As mentioned, the tactile working group was instrumental in the production of an interactive image decision tree. The group worked diligently during Q2 and Q3 to expand on the framework of the original decision tree, which focused purely on image descriptions, and provided final copy for two new branches of the tree with built-in options for tactile graphics and 3d models.
Research Working Group
The research working group was revived late in Q2. Chaired by Dave Edyburn, the group has met once so far and will meet monthly moving forward. Objectives and deliverables are being finalized and will be listed in the Q4 report.
It’s hard to believe that the first quarter of 2020 is already behind us, and yet here we are. While the year has gotten off to a rocky start, to say the least, there are still some bright spots to be found. The DIAGRAM team has embraced all the changes 2020 has thrown at us in our passionate DIAGRAM way, and we’re excited to share some highlights of what we spent Q1 working on. As always thank you all for being a part of our community and we hope you all are staying safe in these turbulent times.
Mathshare, the first-of-its-kind, inclusive, digital math editor that gives all students (with and without disabilities) provides the unprecedented ability to interact with math problems online and to easily demonstrate their math skills in a digital environment. As more classrooms utilize online learning environments, millions of students increasingly face challenges in solving math problems online and showing their work. This quarter we responded to the need for more distance learning tools and added in features that can help parents, teachers and students continue to support math learning from home.
Students are still able to show their work to an educator in a digital, accessible, and user-friendly environment and we added in the ability to collaborate with other learning management systems by completing our integration with Canvas. To help parents support their students at home, we added in the “other” option so that parents and guardians can also support learning through Mathshare. In addition to access to the tool, “other” users will receive automated, targeted emails with instructions on how to get started with Mathshare, how to use it with students, and so on.
Looking forward, our work this quarter focused on building in enhanced personalization features specifically for students with dyscalculia and dysgraphia. Soon, students who struggle with reading will be able to use a text-to-speech option so they can better understand what is being asked in the math problem. Lastly, we started work on allowing students to create their own problem sets which will help them to independently practice math work or copy over examples from other textbooks or worksheets, helping them to prepare for the school year ahead.
Imageshare is an open source platform that enables educators and consumers to find and share multimodal resources related to key STEM concepts. It has been lauded as “desperately needed” and is met with enthusiasm and excitement during user tests and feedback sessions.
In response to the COVID-19 pandemic, this quarter was spent figuring out how to scope Imageshare so that it will be most useful to as many people as possible. The planning undertaken lays the groundwork for a redesign of the Imageshare website that will better support students of all abilities and teachers and emphasize a focus on middle and high school science and math. The redesign will allow us to further support teachers through the expansion of the collection and an increased ability to find the educational supports best suited for their students. If you have files or resources you would like to contribute, please reach out to AmayaW[at]Benetech[dot]org.
The 2020 DIAGRAM report is shaping up to be a strong community effort with experts again volunteering their time and knowledge to write the chapters so you can learn about the featured technology straight from the source!
We would like to extend a huge thank you do this year’s volunteer writers:
Clayton Lewis (DIAGRAM community member, professor of computer science, and fellow of the Institute of Cognitive Science at the University of Colorado) and his colleague, Sidney D’Mello (Associate Professor of the Institute of Cognitive Science) for their chapter on the Internet of Things. Devin Boyle (Senior Consultant at Wheelhouse Group, Emerging Technology Lead at the Partnership on Employment & Accessible Technology (PEAT), and Advisor at XR Access) and her colleague Kai Frazier (CEO of Kai XR and a former history teacher) for their chapter on Mixed Realities. Mario Konecki (Assistant Professor and Advisor to the Rector, University of Zagreb) for his chapter on Accessible Gaming and Volker Sorge (Progressive Accessibility Solutions, Ltd. and Professor at University of Birmingham, UK) for his chapter summarizing how technology for inclusive education has changed over the span of the DIAGRAM + award.
The report has an expected publication date of August 31, 2020 so make sure to stay tuned for additional announcements!
Standards Working Group
The Standards Working Group met several times this quarter, focusing on the development of two new test books: an advanced image description test book and an advanced mathematics test book. The goal is that these books will offer publisher recommendations on how to add math and extended image descriptions accessibility into their content. The group developed content for the image test book and planned for completion and testing in the coming months as well as testing on the advanced math book.
Tactile Working Group
The group continued work on developing an expanded, interactive decision tree that helps users determine the best method for making tactile images accessible. The platform will allow users to answer questions about their objectives with the image in question, and they ultimately receive feedback on whether they should use image descriptions, a 2D model, or a 3D model. A prototype of the interactive interface has been created, and content for the updated tree has been drafted.
Research Working Group
The DIAGRAM Research Working Group is being restarted once again under the talented guidance of Dave Edyburn. This time around the working group will be exploring the areas most related to the work of DIAGRAM, specifically around the need and uses for Mathshare and Imageshare. There will also be space for forward looking research. Look for an announcement of the group’s reformation next month and stay tuned for more information on group specifics and of course, how you can get involved!
2019 was a year of many changes in the DIAGRAM Center. While many of these changes were exciting, some brought mixed emotions. In January 2019, we said goodbye to the Director of Content Acquisition and Core DIAGRAM team member, Robin Seaman, who passed suddenly leaving a hole in many of our hearts. She continues to be missed. In February we also said goodbye to the VP of Labs, Anh Bui, who left Benetech to pursue a passion project with her husband working on satellites. Lastly, we said goodbye to Sue-Ann Ma, who moved to Germany and left Benetech to pursue a different type of passion project with her husband. They welcomed their daughter Maddie in May.
While 2019 brought many losses to the DIAGRAM Center, it also welcomed some new additions. In January, Alex Cabral joined the team as our UX researcher to help make sure we are developing our products for and with input from our targeted audiences. Next we welcomed Matt Nupen to our staff as the senior product manager for DIAGRAM Products, including Mathshare and Imageshare. Additionally, we added a new position to oversee all of the education research and development projects including DIAGRAM. While not new to the team, I assumed the role late in 2019 as the Director of Education R & D. With change brings new opportunities and we are excited to embark on this new year and decade with you.
-Lisa and the rest of the DIAGRAM team
In 2019 we continued to add to and improve the Imageshare tool. We updated the filtering capability and added tag & search resources based on grade level. We also expanded the content collection by partnering with RNIB so Imageshare users have access to their full tactile graphics library. In addition, we added a lot more content giving us a total of 6,000 images in the collection!
For 2020 we will be updating the search feature and how resources are organized so they will be more user friendly. And speaking of user friendly, we will be adding training materials on creating resources. Feature-wise we will be improving the bulk import mechanism and adding user accounts. Stay tuned for status updates as we start working on these new developments.
2019 was a very exciting year for Mathshare. For starters, we launched the beta! Feel free to check it out and see what Mathshare can do for you and your students. If launching the beta wasn’t enough, we also had the opportunity to pilot it with EdReady and a class in Montana for a semester. From there we added updates to make it more screen reader accessible. We also added integrations for Google Classroom and Microsoft Teams so that teachers can use it in conjunction with learning systems they are already using. Lastly, we added personalization features for students with disabilities to modify for their individual needs.
In 2020 we are exploring advanced sketching features and ways to make the sketchpad more usable and accessible for students. We will also be adding grading for assignments and ways for teachers to determine student progress. Make sure to check back for updates on our 2020 progress.
If you are a parent, educator or student, the 2019 DIAGRAM report is written for you. Meant to explore new innovations in technology that have the potential to impact learning, the report focuses on Personalized Learning, Accessible K-12 Computer Science, Multimodal User Interfaces, Accessible Data Visualizations, and Sonification. Each chapter was written by experts in the field, so you are getting the information straight from the source. We here at the DIAGRAM Center would like to extend a huge thank you to Alan Harnum, Richard Ladner, Emily Moore, Jenna Gorlewicz, Doug Schepers and Bruce Walker who authored this year’s chapters.
Work on the 2020 DIAGRAM report is underway. Please keep an eye out for communications about potential topics and other calls to action. The more our community gets involved the richer the report is. We can’t do it without you!
We hope you find the 2019 report interesting, educational, and a valuable resource. Feel free to share it broadly and as always, we welcome all feedback.
2019 saw the passing of two superstars in the accessibility field. At the beginning of the year, we lost Benetech’s Robin Seaman, who passed suddenly on January 7th, 2019, leaving the DIAGRAM community reeling. Jim Thatcher, a disability rights lawyer and pioneer of digital accessibility passed away on December 7th, 2019. His death leaves a huge hole in the hearts of his friends and community. While both Robin and Jim will be missed their legacies will live on. You can read more about Robin in Benetech’s tribute and more about Jim on the tribute published by the law office of Lainey Feingold.