Extended Reality in Astronomy Education/Awareness

Lindsey Gordon holds a first-year Ph.D. in Astrophysics. student at the University of Minnesota Twin Cities. Her past research has focused on exoplanets, quasars and particle accelerators, but she is currently working on computational studies of ICM and AGN jets. If she had free time, she spent it knitting or reading a fantasy novel thick enough to be a door stopper.

Translating the three-dimensional universe into two-dimensional images has always created problems of preserving the size, scale, and positions of objects (see Fluke 2018). This distortion due to the flattening of 3D geometry causes problems throughout astronomy, but particularly for science education, which suffers when learners are confronted with inaccurate data visualizations. Extended reality programs and head-mounted devices (HMDs) may present a new solution to this problem.

A spectrum going from red to left, labeled "Fully digital" in blue on the right, labeled "completely real".  The entire spectrum is labeled as "Extended Reality"with three subsections, from left to right, "Virtual reality", "mixed reality"and "Augmented reality".  Above the spectrum in each region are examples. "Virtual reality" has video games, VR games.  Between Virtual and Mixed is "The void". "mixed reality" has Magic Leap, HoloLens, and ARKit.  Between mixed and augmented is Pokemon Go. "Augmented reality" has Instagram filters and real photos.

Figure 1: Real and virtual environments are often presented on a spectrum ranging from all real at the far right to all digital at the far left. While all enhancements to reality are generally referred to as “extended” reality, most people consider any program with a strong immersive nature to be part of “virtual” reality. [image credit: Northwestern University]

Extended reality is a range of experiences that fall somewhere between a fully real and fully virtual environment, as shown in Figure 1. Depending on the level of immersion, these enhancements may be referred to as mixed reality, augmented reality, or virtual reality (VR), virtual reality being a common buzzword for any type of reality-digital mix that has a strong immersive component. HMDs were once a thing of science fiction, but they’re slowly coming into mainstream use, with commercial offerings from many big tech companies. HMDs provide users with complete visual immersion through a headset component and are typically complemented by one or more wearable devices to control the user experience (see Figure 2 for an example).

A photo of a white male wearing a large black visor over his eyes.  He looks to the right and gestures with two controllers, which are black sticks with holes in the front center.

Figure 2: An example of an HMD setup, with a built-in visor and headphones and two handheld controllers. [image credit: HTC]

Planetariums, helmets and immersion

Extended reality is already present in astronomy education through planetarium shows. Modern planetariums share many characteristics with virtual reality, particularly their preservation of cartography in three-dimensional space and their immersive spherical framing. There are many techniques for developing immersive experiences – Stephanie Rigg The end of the tale is a great resource if you’re looking for more (it’s also a beautiful book). The planetarium and headset experiences share many similarities, including using screens that occupy the entire view of the audience, creating programs that emphasize audio/visual cues over text, and developing a ritual in space. The same way you walk into a planetarium and experience pre-show cues like dimming lights and announcements, in a headset you have a ritual of putting on gear and watching a loading screen. Creating a smooth transition between full reality and a mixed reality experience is a key part of the immersive experience for audiences. The agency of the audience to be explored is often touted as one of the benefits of a VR experience; users make their own decisions to create a self-directed experience that can have more impact on their education. Sophisticated planetariums offer live broadcasts that allow viewers to interact with the presenter, ask questions, and guide the visual and auditory experience of the broadcast. Education is a goal for planetariums, but often the goal of science museums is to be the impetus for future exploration by visitors; they teach viewers that it is possible to have fun in a scientific setting to encourage continued curiosity.

A photograph of the interior of a planetarium with a huge domed screen currently showing a region of bright red and orange gas.  Beneath the screen, rings of slant-backed chairs all face a central podium.

Figure 3: The Planetarium at the Boston Museum of Science. Note the placement and angles of the large screen and chairs, with the seats angled back to immerse guests as much as possible by making the most of the screen, if not all of their view. [image credit: Boston Museum of Science]

The problem with planetariums is that they’re big, expensive to build, and (usually) immobile. Professional observatories and telescopes are also usually stationary and are also limited by location, weather and cost. An HMD is relatively inexpensive – while current models cost several hundred dollars, they can be used by many guests over years of outreach as a long-term investment with a lower cost per use than planetarium tickets . HMDs are also portable, requiring only a computer to connect, and accessible, with very few physical requirements for their use. Like a planetarium, an HMD-based VR program can preserve three-dimensional spatial relationships, visualize large datasets, and create an immersive user-directed astronomy experience. Unlike a real observatory, they are not dependent on weather, local light pollution, or physical location on Earth. HMDs are more commercially available than ever, but are not yet a household staple. This offers the advantage of being something new and interesting for customers to discover; there is a “wow factor” to their inclusion in educational events. Guests have been known to queue for over an hour to try out very short HMD-based astronomy experiments.

On the other hand, their use requires funding to acquire the device(s), as well as specialized software and training for educators offering HMD programming. There’s also a question of the isolated nature of headset use. Although the immersion may be the same or better at an HMD, the common experience of going to a planetarium show is considered psychologically important to the experience. The sense of community is notorious for being difficult to recreate in VR; virtual group gatherings are not the same as being all in one place. Common VR experiences would also require observatories and classrooms to purchase enough headsets for all users to enjoy the experience at the same time, leading to high costs. However, this common effect may not be necessary, since the use of the HMD itself as an emerging technology may generate similar enthusiasm towards the subject presented.

A screenshot of WWT's desktop view.  It shows the Earth in the distance, lit on the left side by the sun, and the Moon is labeled in the background although not visible.  The top and bottom menus show the other planets in our solar system as options to explore.

Figure 4: Desktop view of WWT, centered on a view of Earth. The desktop version of WWT still suffers from 3D to 2D translation, but it’s a free and accessible way to explore real astronomical data. [image credit: WWT]

VR programs available to the public

Despite the difficulties, there has been a boom in the development of software for teaching astronomy in virtual reality. The most important software released to date is that of Microsoft world telescope (WWT) platform, which was later taken over by the American Astronomical Society (AAS) (WWT article hereWWT online version here). WWT presents a spatially accurate view of all publicly available astronomical catalog data and allows users to explore everything from our solar system to deep space in a browser, desktop (see Figure 4) or projected in 3D in a planetarium or HMD. WWT has been successfully integrated into a number of smaller planetariums, K-to-university classrooms, and VR-based museum exhibits. WWT shows and classic planetarium shows use what’s called a “contextual narrative layer,” a VR user experience technique that uses storytelling, user-driven exploration, and embedded real-world information (to learn more about the specifics of this technique as implemented in WWT, see Wang 2008). Users can then follow recorded 3D tours or explore on their own, while having access to the actual catalog data used to generate their view. WWT is free and open source and is designed to be customized for different user experiences, making it one of the most popular tools for teaching VR astronomy.

Other virtual reality implementations for astronomy education include allSkyVR (paper; Software), released in 2018, which was designed as an out-of-the-box astronomy visualization tool for HMD-based virtual reality as well as a basis for customizable development. Stellarium (paper; Software) is a tool similar to WWT that is designed primarily for desktop use, but also has extensions for planetarium projection use. They also have an augmented reality mobile app that uses location data from the phone to display an overlay of what’s in the sky at the time. MarsVR (paper; website) is a recent launch that uses real NASA Martian data to allow users to explore the surface of Mars and is explicitly designed for public education and HMD use. Astrophysical Big Bang Laboratory scientists produced the VR experiment SN2SNR (paper) to allow guests to explore simulations of a supernova explosion and its transition into a supernova remnant.

Virtual reality programs are slowly beginning to take root in community astronomy education. There are no perfect educational tools, but HMD-based virtual reality programs have many advantages that observatories, museums and classrooms are only beginning to exploit.

Edited by Macy Huston & Jenny Calahan
Featured image of Boston Science Museum

Comments are closed.