AAA

3D printing makes virtual world more real for blind learners

Michael A. Kolitsky

Transition of the real world into the virtual world has been ongoing in online and on-ground classrooms and labs for many years. But the more the world goes virtual, the more difficult it becomes for those who are blind or visually impaired to participate fully in the learning process. The main thrust of this report is to explore how 3D printing and the use of conductive filament can begin to reverse that process by making 3D printed talking tactile learning objects that can generate audio when touched after being placed on the surface of an iPad or other tablet computer. Examples of talking tactile learning objects from STEAM (Science, Technology, Engineering, Arts and Math) disciplines will be presented with a focus on how the design of 3D printed STEAM learning objects can lead to a common learning object template model. The template can be designed to serve as a learning tool not only for a blind or visually impaired student but also can work as well for a sighted student or a teacher, aide or parent demonstrating how universal design principles can drive the learning object production process.

Virtual reality, video and complex images all contribute to making access to the virtual digital world more difficult for students who are blind or visually impaired. Efforts to make virtual reality more accessible by using audio are ongoing as shown in the use of 3D printed touch-responsive maps that can vocalize information about buildings and directions1. Another earlier attempt2 was designed to permit an audio-augmented experience for exploration of virtual environments that simulate exploring a museum. The use of audio has also been emphasized in the design of AudioMath3, a virtual environment designed for blind children to learn mathematical concepts. Audio-enhanced learning was also utilized in a museum exhibit at Pine Knoll Shores Aquarium (Winston-Salem, North Carolina) titled Fintastic! Exhibit and was the first of its kind museum exhibit to display tactile talking fish models so that blind or low vision visitors could begin to experience by touch what sighted visitors see4. A white paper5 by RAF Models reported research on the Fintastic! Exhibit showing learning was dependent upon a tight link between the audio descriptions of fish models and the shape, form and textures on those models. In this report, 3D printed audio-enhanced tactile learning objects were made using conductive filament so that a 3D print could then be laid on top of an iPad or other tablet computer for audio production when the 3D print was touched by the learner's finger. Audio inclusion in 3D printed tactile learning objects by using commonly available iPads or tablets expands options for designing new ways to learn about virtual worlds especially for students who are blind or visually impaired.

Making tactile learning objects talk

For a 3D print to become a learning object, it must be integrated into a stand-alone, reusable chunk of learning that traditionally was also in digital format as shown by the many thousands of learning objects now available for downloading from repositories like Merlot6. In order to make a 3D printed talking tactile learning object, the combination of a digital component and a real 3D printed object had to be constructed. The real portion was the 3D print and the digital component required for the audio response was provided by an iPad for this report but other computers such as the Microsoft Surface or Android tablets were tested and could work as well. Conductive 3D print filament from BlackMagic3D7 containing graphene was also needed to carry the electrical charge from a fingertip touch on the 3D print to the iPad surface so that some areas on the 3D print that were made for touch sensitivity could transfer a strong enough fingertip electrical charge to change the capacitance of the iPad screen on which the 3D print was placed. The final piece needed to make a functional talking tactile learning object centered on an HTML5 3D graphics app named „Hype 3”8 by Tumult which permitted the design of iPad audio buttons that could be positioned directly underneath the conductive filament so that a fingertip touch would now cause the iPad button to generate an audio response.

Hype 3 permits the iPad screen graphic to closely replicate the 3D print so that it can also serve as a learning tool by sighted students without the 3D print meeting the goals of universal design pedagogy9. Although Hype 3 is designed for the Mac OS X 10.8+ operating system, the produced HTML5 content is web deliverable and runs on all modern browsers and tablet computers.

A Makerbot Replicator 210 was used to make a 3D printed „holder” designed to hold the iPad in place so that a plastic sheet with 3D printed objects made from conductive filament attached to it could be placed on the iPad screen surface and held so that areas of conductive filament were located directly above the software audio buttons on the iPad. Some software adjustments in the HTML5 code of Hype 3 were also made to inhibit screen movement on the iPad so that audio buttons were always in a location directly underneath the conductive filament portion of the 3D print. This combination of thin plastic sheet cut to fit in the holder along with the 3D printed material that was attached to the top of the thin plastic sheet was called a template. The thin plastic sheet was cut from file folders and did not inhibit touch activated iPad audio production.

Complex image templates

The author11 and others12 have begun to show that 3D printers can make 3D prints of microscope slides, telescope photos and earth-facing satellite images that can be used as a starting point for making tactile learning objects. 3D printing is also changing the way teachers teach and students learn across many disciplines13. One program valuable for making 3D print stl files for 3D print production is „PhotoToMesh”14 which can make 3D prints that are in „bas relief” form with one side flat and the other side designed for providing a tactile function. In the template below in Figure 1, a bar graph comparing use of practice quizzes in an Anatomy and Physiology course was used as an example of a complex graphic. The stl file produced by PhotoToMesh was designed so that one data set had bars raised above the background level to a different level than the other comparable data set and also the data sets were designed to be different by making them with different patterns able to be discerned as different when touched. The upper region of each bar graph was made with conductive filament making it capable of carrying an electrical charge from a fingertip touch that can then cause the audio buttons on the iPad to generate audio, i.e., „talk” as in talking tactile learning object. An audio button made with Hype 3 was placed on the iPad screen just beneath the 3D printed conductive filament so that audio is produced when a finger touches the top of the bar graph. The audio presents information about that bar graph component such as its xy location, the percentage value on the vertical axis and what grade category it represents. This is also an example of a way of making a complex image more discernable via touch and audio delivery for the blind learner as well as for a sighted student fitting what is expected for designing by universal design standards.

The left side of Figure 1 below shows an iPad with a complex graphic image of a bar graph comparing the percentage of students receiving A, B, C, D and F grade when credit was given or not given for doing practice exams15. The 3D printed bar graph was designed for an invited presentation made on the main campus of Penn State University in the fall of 2015 to demonstrate the way that conductive filament could be added for an audio-enhanced tactile experience. The right side of Figure 1 shows the iPad now with a 3D printed complex graphic template on a thin blue plastic sheet added to the iPad surface being held in place by a 3D printed plastic holder. For a more complete learning experience, there would have to be additional buttons made from conductive filament such as the star shaped symbol in the upper left of the template so that information about what the graph represents could be delivered when that conductive button was touched. Also note the 3D printed triangle below the 3D printed bar graph which when touched provides an audio description of the results shown on the 3D printed bar graph. The dark rectangular areas at the top of each bar are composed of conductive filament and when touched, provide an audio response giving the y axis value in percent of students receiving a particular grade as well as whether the bar represents students who received or did not receive credit for doing the practice quizzes.

Figure 1. 3D printed bar graph in template form

Source: own study.

Example of video template based on NASA moon transit of Earth

During the summer of 2015, NASA published some spectacular videos showing the moon making a transit across the surface of the Earth16. The videos were made from images taken by the EPIC camera on the DSCOVER satellite parked in stationary orbit one million miles from Earth. The unique nature of this video was that the moon was being viewed from the dark side as it was making its transit and this side of the moon is never seen from the Earth. Can a blind or visually impaired viewer sense the excitement of this video when the video cannot be seen? One approach has been to make 3D prints of the moon in various stages of transit and attach these isolated 3D prints to the plastic template with audio buttons underneath on the iPad surface so that by finger touch, a blind student can sense and also hear reinforcement of what is being touched by a corresponding audio response. Discussion with a NASA engineer17 also indicated that a template could be designed to measure the speed of the moon in its orbit using the diameter of the Earth and the moon in the calculations that would add a real world math component to the moon transit template that appears below in Figure 2.

Left portion of Figure 2 below shows the iPad graphic and partial view of a 3D print on a clear thin plastic overlay. Right side of Figure 2 below shows 3D print on surface of iPad.

Figure 2. NASA moon transit of Earth

Source: See reference 16 for source of images made into 3D prints.

By touching the dark circles on right image, an audio response is generated indicating that the moon is in a particular phase of moon transit. The rounded star on upper left provides audio response describing what the template is showing and the small triangles below the three transit phases provide an audio response indicating what stage the transit is in as it passes the Earth's surface.

Virtual world template

Several decades ago, the national Library of Medicine completed the Visible Human Project18 in which a male cadaver was frozen and sectioned to produce over 1800 transverse sections that have been used by many universities and publishers to make highly interactive virtual reconstructions for analysis using imaging algorithms. But, again, how do blind students wishing to learn anatomy utilize these virtual resources? One recent example was demonstrated by the author at the regional meeting of the American Association of Anatomists Conference in London, Ontario in which a poster presentation was made showing that 3D prints of a transverse section of the head that had conductive filament embedded in the 3D print at locations of the right eye, optic nerve and optic chiasma could now generate an audio response when placed on an iPad and touched in the region of the conductive filament19. A testing component was also developed for this template so that incorrect answers generated a wrong answer response and correct answers were confirmed with an audio response.

Left side in Figure 3 below shows iPad graphic and partial view of 3D print on clear plastic overlay. Right portion of Figure 3 below shows 3D print template on surface of iPad. By touching the dark conductive filament on the complete template, an audio response is generated indicating what structure was being touched.

Figure 3. Virtual anatomy template

Source: own work.

Reading poetry in 3D space template

An article published in the English Journal20 showed how four Haiku with the same middle line and written by four different students could now be configured in 3D space so that the four Haiku could arrange themselves in a virtual cube of poetry with first and third lines of each Haiku now positioned along the exterior of a poetic cube. Sighted students in an upper division creative writing class exhibited an „ah ha” moment when for the first time the edge lines on the poetic cube formed from the first and third lines of the four Haiku could now be read to produce poems which were not able to be seen until arranged into the 3D virtual poetic cube. Figure 4 below on the left shows one view of a 3D Haiku cube and the image below on the right shows the same 3D Haiku cube rotated to the right further clarifying the position of the nine lines composing this 3D Haiku cube. The Haiku lines represented in the 3D poetic cube are listed below the 3D Haiku cube rotation views. Note the value of being able to arrange the four Haiku into a poetic cube since it is only when in 3D cube arrangement that the exterior end lines can be read as poems.

Figure 4. Two views of 3D Haiku cube in virtual space

Source: author's own work.

Figure 5 below on the left shows the iPad with a Hype 3 graphic based on viewing a 3D Haiku cube from the side where the nine Haiku lines would each begin. Touching a white rectangle generates audio indicating which line is represented in the 3D Haiku cube. For example, touching the upper right white rectangle would cause the audio button to respond „Haiku 1 Line 1” and touching the middle white rectangle would cause the audio button to respond „Haiku 1 Line 2”. When the white button on the lower left is touched, the audio button would respond „Haiku 1 Line 3”. The image on the right shows the Haiku reading template added on top of the iPad surface. The rectangular black buttons made with conductive filament are located directly over the white audio buttons on the iPad graphic. With a new Hype 3 page loaded onto the iPad, touching the rectangular black buttons composed of conductive filament now cause the buttons on the iPad surface to generate the Haiku line related to the position of the button in the poetic cube. For example, touching now the button in the upper right corner of the 3D Haiku cube would generate the line „Wind begats waves white caps” and touching the middle button would generate „Dreams smith beachwalk hand-in-hand”.

Figure 5. Reading 3D Haiku lines by generating audio using touch

Source: own work.

Value of templates in universal design

For all the templates described in this paper, the image on the iPad screen is an exact duplicate of the 3D printed template meaning that a sighted person could also utilize the learning object but without the 3D printed template. This is of importance for teachers, support staff and parents who may be assisting a blind student in the learning process and also for sighted students who are learning in the same classroom. When a learning tool can be used for learning by both sighted as well as blind or visually impaired students, one of the basic requirements has been met for applying universal design principles to the development of talking tactile learning objects.

Availability of audio-enhanced 3D printed templates

The templates described in this article can serve as examples of how images from complex graphics, videos and virtual worlds can be made into audio-enhanced tactile learning objects using entry level 3D printing skills and some knowledge of design software. Many academic institutions are now forming maker communities to assist faculty in the production of audio-enhanced 3D printed learning objects. A critical component though in the future application of audio-enhanced 3D printed learning objects is testing their effectiveness in the learning process. In this regard, the author is interested in developing collaborative opportunities that would involve the sharing of 3D print files for production of learning objects such as now exist at the NIH 3D Print Exchange21 and where the author has listed several 3D print stl files for the study of mitosis22. All 3D prints in this paper and ways to make audio-enhanced tactile learning objects are available without cost to colleagues for testing with the goal of sharing data for the improvement of both the design process and its impact on student learning.

References

INFORMACJE O AUTORZE

Michael A. Kolitsky

Michael A. Kolitsky received his Ph.D. from Temple University in Philadelphia, PA. and is now retired but teaching online for The University of Texas at El Paso (UTEP) and Atlantic Cape Community College (ACCC). Mike was a tenured professor of Biology at UTEP and also was appointed Associate Vice President for Instructional Technology overseeing the technology design and faculty training for a new Undergraduate Learning Center. He was the Principal Investigator for a 2.5 million dollar NASA grant to establish an instructional support and training center at UTEP for University and regional K-12 instructors. Mike also received a University of Texas Chancellor's Distinguished Teaching Award at UTEP and earlier was awarded a Distinguished Natural Sciences Curriculum Innovation award for his Embryology videodisc and HyperEmbryo courseware from EDUCOM. After leaving UTEP, he was appointed a founding board member for NJEDge.Net, the higher education network for New Jersey and served as Dean of Academic Computing and Distance Education at ACCC prior to retirement. Mike was a consultant for the Rutgers University Library FIPSE grant to establish the New Jersey Digital Highway and also assisted in the production of several shared content objects (SCO's) with The University of Wisconsin-Madison Academic ADL Co-Lab FIPSE-sponsored effort to support effective online teaching. Mike is interested in the future of mobile computing as seen in a conference presentation he made in May, 2008 when his iPhone was utilized in place of a laptop for presentation graphics.

 

Informacje o artykule

DOI: 10.15219/em63.1222

W wersji drukowanej czasopisma artykuł znajduje się na s. 65-70.

pdf pobierz artykuł w wersji PDF

Cytowanie

Komentarze

Nie ma jeszcze komentarzy do tego artykułu.

dodaj komentarz dodaj komentarz

Przypisy

1 A 3-D talking map for the blind (and everyone else), ScienceDaily, 2014, www.sciencedaily.co.... [19.02.2016].

2 A. Zimmermann, A. Lorenz, LISTEN: A User-adapted Audio-augmented Museum Guide, „The Journal of Personalized Research” 2008, Vol. 18, No. 5, pp. 389-416, dx.doi.org/10.1007/....

3 J. Sanchez, H. Flores, AudioMath: Blind Children Learning Mathematics Through Audio, „International Journal on Disability and Human Development” 2005, Vol. 4, No. 4, pp. 311-316, dx.doi.org/10.1515/....

4 Alliance of Disability Advocates, www.alliancecil.org....

5 R. Fuller, W.R Watkins, Exhibit Design Relating to Low Vision and Blindness (White paper), Indiana University, Bloomington 2010, www.ncaonline.org/d.... [19.02.2016].

6 Multimedia Educational Resource for Learning and Online Teaching, https://www.merlot..... [19.02.2016].

7 BlackMagic3D, www.blackmagic3d.co.... [19.02.2016].

8 Tumult, tumult.com/hype/. [19.02.2016].

9 National Center on Universal Design for Learning, www.udlcenter.org/. [19.02.2016].

10 MakerBot, https://www.makerbo.... [19.02.2016].

11 M.A. Kolitsky, 3D Printed Tactile Learning Objects: Proof of Concept, „Journal of Blindness Innovation and Research” 2014, dx.doi.org/10.5241/....

12 E. Hasper, et al., Methods for Creating and Evaluating3D Tactile Images To Teach STEM Courses to the Visually Impaired, „Journal of Science Teacher Education” 2014, Vol. 4, No. 6, pp. 92-99.

13 M.A. Kolitsky, Reshaping teaching and learning with 3D printing technologies, „e-mentor” 2014, pp. 84-94, dx.doi.org/10.15219....

14 Ransen Software, www.ransen.com/phot.... [19.02.2016].

15 Source: own study.

16 From a Million Miles Away, NASA Camera Shows Moon Crossing Face of Earth, NASA, 2015, www.nasa.gov/featur.... [19.02.2016].

17 Personal correspondence with Dr. Adam Szabo, DSCOVER Project Scientist, NASA Goddard Space Flight Center.

18 The Visible Human Project, NIH National Library of Medicine, https://www.nlm.nih.... [19.02.2016].

19 M. Kolitsky, 3D printing makes Visible Human cadaver sections accessible for blind students, www.nextgenemedia.c.... [19.02.2016].

20 S. Tweedie, M.A. Kolitsky, 3-D Haiku: A New Way to Teach a Traditional Form, „English Journal” 2002, Vol. 91, No. 3, pp. 84-88.

21 NIH 3D Print Exchange, 3dprint.nih.gov/. [19.02.2016].

22 NIH 3D Print Exchange, M.Kolitsky, Stages of mitosis, 2014, 3dprint.nih.gov/dis.... [19.02.2016].