Virtual and Mixed Reality and the future of mathematical practice

Abstract

In the summer of 2015, the speaker led a team at Microsoft Research, comprised primarily of research interns from seven universities, to demonstrate a social, ambulatory, Mixed Reality system for the first time.  Each intern developed a preliminary, domain-specific exploration of how such a system could be used.  One of the interns, Andrzej Banburski of the Perimeter Institute, demonstrated an interface to Mathematica.  The very concept and terminology of Virtual Reality had been first articulated by the speaker almost four decades earlier precisely in the hopes of experiencing this type of interface.  Mathematical expressions floating in the air is not just more vivid that writing them on a blackboard; a different workflow and mental framework becomes possible.  For instance, functions appear as lenses that can be compounded as they are lined up.  One can peer through a stack of such lenses at different data or starting conditions.  Instead of erasing and replacing portions of expressions on a blackboard, a stack of lenses can fork and merge, so that a range of variations can be explored at once as aspects of a sculptural form.   The purpose is not just to make math more accessible to those who might find this interface more inviting, but to free math from pre-computational, notation-bound conventions.  The speaker will argue that this type of interface, while still only barely explored, could eventually have a significant impact both on computer science and other disciplines, including theoretical physics.