June 2012 Software Development Update
Tools were a major theme of the DIAGRAM Asilomar meeting this month, with many ideas emerging for new reading and authoring tools, including reference implementations of DIAGRAM standards. We’ll be working through the ideas in order to refine our development road map. Over the next few months we’ll also get progress updates on our development subcontracts, as well as new feature releases for Poet as work that is currently underway gets completed.
Benetech continues to improve Poet in order to make it easier for describers to tackle some of the most difficult content. This summer that has meant math, math, and more math, consistent with our focus on STEM content and with the emphasis on math coming out of Asilomar. We have several projects on this front, some of which are funded out of other initiatives at Benetech but are of keen interest to DIAGRAM. Most recently, Benetech’s engineering team has been doing some final testing on a feature in Poet called Math Helper, which automatically takes ascii math expressions input by a describer and turns them into MathML in real time. Math Helper also converts the input into instant,typeset display in the browser using MathJax, so that describers can check their work. The feature will make it much easier for describers to create machine-readable math expressions without having to learn the notoriously difficult MathML syntax.
The new Math Helper feature takes advantage of work done by volunteers at a GoogleServe hackathon event in early June, more evidence that harnessing the power of social coding can result in some amazing capability in a very short amount of time. Another hackathon in June, a Random Hacks of Kindness event, resulted in an algorithm that automatically determines whether a digital image is a mathematical expression or some other image. This will allow us to someday process images in advance and tag the ones that are equations to behandled separately, perhaps transcribed by something like Math Helper. Moreinfo on the math images identifier can be found at this link: https://docs.google.com/a/benespace.org/presentation/d/1WaQZtOKb_AZ7nIFWCiIBhe8PGboMAHIraAtlOMb6tts/edit#slide=id.p
Finally, Benetech is making progress advancing NASA’s Math Description Engine for use in creating accessible math. Back in April, Benetech used the NASA Space Apps Challenge to reinvigorate NASA’s MathTrax, a graphing tool for middle school and high school students to graph equations, physics simulations or plot data files. The graphs produced by MathTrax are accompanied by automated text descriptions and sonification. The open source Math Description Engine (MDE) underlies MathTrax and can be harnessed to provide automated math description services to tools like Poet. Working with one of the original developers of the MDE, we are putting the engine to work on an Algebra 1 textbook in order torefine its descriptions and outputs.
Putting all of this work together, we can imagine taking a book, running it through a process to identify all the image-based mathematical expressions, having those expressions converted into MathML, having the MathML converted automatically into diagrams and descriptions where necessary, and then having the math beaccessibly displayed and voiced.