A Design Space for Digital Augmentation of Reading

Pedro Ribeiro, Wolfgang Müller, Ido Iurgel, Christian Ressel and Carrie Ching

Rhine-Waal University of Applied Sciences, Kamp-Lintfort, Germany
University of Education Weingarten, Weingarten, Germany
pr@hsrw.eu

Abstract.

To comprehend a text the readers can benefit from their capacity to bodily interact with the environment to construct the meaning of a text. Specifically, they use their perception, action, and emotion systems to gain a deeper understanding of the text. Readers’ mental simulations of perceptions, actions, and emotions associated with a text facilitate reading comprehension, motivation and learning. The digital augmentation of reading aims at the artificial enrichment of reading through digital media. This concept has the potential to digitally synchronize the text with the surrounding environment and to promote an embodied reading experience. This paper reviews and analyses recent relevant work about the digital augmentation of reading. It aims to construct a design space to support researchers, designers or educators to systematically explore design parameters that enable the effective creation of embodied reading experiences, which foster comprehension and learning. This research indicates the under-explored configurations of the design space as well as the most frequently used configuration patterns. The paper also provides examples of how the design space is utilized (e.g., in school).

 

References

  1. Glenberg, A.M.: Embodiment as a unifying perspective for psychology. WIREs Cognitive Science. 1, 586–596 (2010). https://doi.org/10.1002/wcs.55.
  2. Grasset, R., Dunser, A., Billinghurst, M.: The design of a mixed-reality book: Is it still a real book? In: 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality. pp. 99–102 (2008). https://doi.org/10.1109/ISMAR.2008.4637333.
  3. Campos, C., Ducasse, J., Čopič Pucihar, K., Geroimenko, V., Kljun, M.: Augmented Imagination: Creating Immersive and Playful Reading Experiences. In: Geroimenko, V. (ed.) Augmented Reality Games II: The Gamification of Education, Medicine and Art. pp. 57–81. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3-030-15620-6_3.
  4. Cardoso, J.C.S., Ribeiro, J.M.: Tangible VR Book: Exploring the Design Space of Marker-Based Tangible Interfaces for Virtual Reality. Applied Sciences. 11, 1367 (2021). https://doi.org/10.3390/app11041367.
  5. Milgram, P., Kishino, F.: A taxonomy of mixed reality visual displays. IEICE TRANSACTIONS on Information and Systems. 77, 1321–1329 (1994).
  6. Brudy, F., Holz, C., Rädle, R., Wu, C.-J., Houben, S., Klokmose, C.N., Marquardt, N.: Cross-device taxonomy: Survey, opportunities and challenges of interactions spanning across multiple devices. In: Proceedings of the 2019 chi conference on human factors in computing systems. pp. 1–28 (2019).
  7. Levin, M.: Designing Multi-Device Experiences: An Ecosystem Approach to User Experiences Across Devices. O’Reilly Media, Inc. (2014).
  8. Qi, J., Buechley, L.: Electronic Popables: Exploring Paper-based Computing Through an Interactive Pop-up Book. In: Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction. pp. 121–128. ACM, New York, NY, USA (2010). https://doi.org/10.1145/1709886.1709909.
  9. Yamada, H.: SequenceBook: Interactive Paper Book Capable of Changing the Storylines by Shuffling Pages. In: CHI ’10 Extended Abstracts on Human Factors in Computing Systems. pp. 4375–4380. ACM, New York, NY, USA (2010). https://doi.org/10.1145/1753846.1754156.
  10. Rahman, A.S.M., Alam, K.M., Saddik, A.E.: Augmented HE-Book: A Multimedia Based Extension to Support Immersive Reading Experience. In: Autonomous and Intelligent Systems. pp. 321–330. Springer, Berlin, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21538-4_32.
  11. Schafer, G.J., Fullerton, S.K., Walker, I., Vijaykumar, A., Green, K.E.: Words Become Worlds: The LIT ROOM, a Literacy Support Tool at Room-Scale. In: Proceedings of the 2018 Designing Interactive Systems Conference. pp. 511–522. Association for Computing Machinery, New York, NY, USA (2018). https://doi.org/10.1145/3196709.3196728.
  12. Silva, E.P., Vieira, N., Amorim, G., Mousinho, R., Guedes, G., Ghinea, G., Dos Santos, J.A.: Using Multisensory Content to Impact the Quality of Experience of Reading Digital Books. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM). 17, 1–18 (2021).
  13. Yoshino, K., Obata, K., Tokuhisa, S.: FLIPPIN’: Exploring a Paper-based Book UI Design in a Public Space. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. pp. 1508–1517. Association for Computing Machinery, New York, NY, USA (2017).
  14. Billinghurst, M., Kato, H., Poupyrev, I.: The MagicBook: a transitional AR interface. Computers & Graphics. 25, 745–753 (2001). https://doi.org/10.1016/S0097-8493(01)00117-0.
  15. Figueiredo, A.C., Pinto, A.L., Branco, P., Zagalo, N., Coquet, E.: Bridging Book: A Not-so-electronic Children’s Picturebook. In: Proceedings of the 12th International Conference on Interaction Design and Children. pp. 569–572. ACM, New York, NY, USA (2013). https://doi.org/10.1145/2485760.2485867.
  16. Vogelsang, A., Signer, B.: The lost cosmonaut: an interactive narrative environment on the basis of digitally enhanced paper. In: International Conference on Virtual Storytelling. pp. 270–279. Springer (2005).
  17. Gordon, G., Breazeal, C.: Bayesian Active Learning-based Robot Tutor for Children’s Word-reading Skills. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. pp. 1343–1349. AAAI Press, Austin, Texas (2015).
  18. Sanchez, S., Dingler, T., Gu, H., Kunze, K.: Embodied Reading: A Multisensory Experience. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. pp. 1459–1466 (2016).
  19. Grasset, R., Duenser, A., Seichter, H., Billinghurst, M.: The mixed reality book: a new multimedia reading experience. In: CHI’07 extended abstracts on Human factors in computing systems. pp. 1953–1958 (2007).
  20. Back, M., Cohen, J., Gold, R., Harrison, S., Minneman, S.: Listen Reader: An Electronically Augmented Paper-based Book. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 23–29. ACM, New York, NY, USA (2001). https://doi.org/10.1145/365024.365031.
  21. Cingel, D., Blackwell, C., Connell, S., Piper, A.M.: Augmenting Children’s Tablet-based Reading Experiences with Variable Friction Haptic Feedback. In: Proceedings of the 14th International Conference on Interaction Design and Children. pp. 295–298. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2771839.2771900.