The Virtual and the Real: Intangible Cultural Heritage and Holograms

Origin and History of Holograms

Korea is currently experiencing a Fourth Industrial Revolution. And there are increasing attempts to adapt the Fourth Industrial Revolution in cultural heritage across the country, and holograms are recently drawing keen attention as a potential technological advancement for this purpose. However, hologram technology is nothing new; rather, it can be traced back to 150 years ago.

Hologram is a newly coined word comprised of the Greek words holos, meaning “whole,” and gramma, meaning “painting.” In sum, the term refers to a perfect painting that shows not just one side of an object, but its entire shape simultaneously. The theatrical technique of Pepper’s Ghost, which was introduced in the late nineteenth century, created impeccable stereoscopic images that were projected at the front of the stage, but the illusion was exposed when viewing from the side or back of the stage.

A scene from Pepper’s Ghost that was all the rage across Europe in 1862.

Therefore, the simple optical illusion of Pepper’s Ghost failed to create a perfect painting, strictly speaking. Nonetheless, in hindsight, it was the prototype of pseudo-holograms, given that it created an impeccable glasses-free, stereoscopic 3D image in the eyes of the audience, by applying the principle of a 45-degree angle reflector.

As mentioned above, it was approximately 150 years ago when humankind successfully set an image afloat in three-dimensional space for the first time. Later, in the late nineteenth century, an English engineer named Henry Dircks invented a remarkable image technique to levitate a ghost onstage. Using this technique, a bright light was focused on the subject from a dark room under the stage. Its image was reflected onto a mirror inclined backward at a 45-degree angle and then projected onto a glass plate installed at a slanted angle onstage. In the eyes of the audience, a ghost appeared as if it was floating in the air, as the shape of the subject was reflected onto a transparent film layered over the dark background.

Ultimately, this technique created an optical illusion by utilizing the simple principle that a glass plate installed in a dark place would reflect light, just like a mirror. After several years had passed, John Pepper staged a ghost performance in a theater using a novel optical technology. This show, titled Pepper’s Ghost, soon became a sensation across European theaters. In a way, the ghost-performance pattern designed by Henry Dircks laid the theoretical foundation for the pseudo-hologram described above and laid down the framework to create fabulous holographic effects in large performances and state-of-the-art digital shows today.

As optical or laser holograms are no longer in the spotlight today, Pepper’s Ghost can be seen as the origin that gave birth to the holograms with industrial applications.

It was only during the twentieth century that a genuine holographic technology, as opposed to an optical illusion, was developed to generate stereoscopic light flows by projecting light beams onto an actual space.

In 1947, the English physicist Dennis Gabor originally discovered holography that generates stereoscopic images of an object onto a three-dimensional space through the intersection of two light sources, and this was later acknowledged as a great invention that earned him a Nobel Prize.

Pseudo-holograms Applied in Cultural Heritage

These so-called pseudo-holograms are not genuine holograms from the viewpoint of optical holograms. Instead, they reflect objects using computers-based projectors. Pseudo-hologram is translated to “hologram-like” in English and “擬似 (similar to reality) hologram” using Chinese characters.

Pseudo-holograms are shown by reflecting actual figures and images into the air using special displays, and special beam projectors and LEDs are used to generate stereoscopic images reflected in the air. Its principle began with the technique of Pepper’s Ghost whose trial performance was staged across Europe in 1862.

Until recently, it had been referred to as a 3D holographic projection as a similar concept, while the term “pseudo-hologram” was coined to attach the concept as a type of hologram as a matter of convenience. Representative cases for the commercialization of this pseudo-hologram include Musion’s Eyeliner, Vizoo’s Free Format, Lincoln Museum’s mist hologram exhibition, Actuality Systems’ Perspecta, Activ 3D’s 3D-holobox hologram display, InnoVision Labs’ HoloAd and Realfiction’s Dreamoc XL.

Cases of Exhibitions for Korean Cultural Heritage Using Hologram Technology

A large number of cases where holograms are used for commercial purposes are those based on the Pepper’s Ghost technique of projecting three-dimensional images onto a large transparent screen. It remains unfeasible for individuals to enjoy holographic illusions at home, but large-scale holograms are beginning to be used by institutions with the necessary technological and financial resources, such as concerts by famous stars and major companies’ marketing events.

1) Digilog Samulnori

An awe-inspiring performance titled, Digilog Samulnori: The Dead Tree Blooms, was held at Gwanghwamun Art Hall in January 2010. It was the world’s first attempt at combining samulnori, or Korean traditional percussion music, with 3D holographic projections. Scripted by Lee O-young, the former Minister of Culture, this new style of artistic performance gained tremendous attention as a collaboration between performers and holographic characters. Kim Duk-soo, a master of samulnori, collaborated with his holographic copies, accompanied by Korean traditional dance and songs performed by traditional choreographer Guk Su-ho and pansori singer Ahn Sook-sun.

As a novel format of performance based on the combination of traditional culture and advanced technology, Digilog Samulnori was inspired by the idea of associating samulnori—which represents a traditional flow of nature across the seasons of spring, summer, fall and winter, along with rain, cloud, thunder, and wind—with 3D projection technologies. To describe this amalgamation, the term “digilog” was coined as a portmanteau of “digital” and “analog.”

The holographic storytelling behind the performance is as follows. As the curtains are raised, a dead tree in the desert comes into view to symbolize industrialization. Subsequently, samulnori is performed in harmony with a large gong, an hourglass-shaped drum, a small gong, and a barrel drum, bringing vivacity to the scene of the dead tree while the accompaniment of samulnori led by Kim Deok-su, dance by Guk Su-ho, and singing by Ahn Sook-sun, with fantastical holographic illusions amplifies the sensory experience of the audience.

Amid the rhythms of traditional percussion instruments resonating through the hall, the stage presents surreal images, such as the dead tree that abruptly blooms, three or four virtual Kim Deok-su copies each playing a different musical instrument. As such, the dimensional illusions projected onto the stage in this performance delivered a shockwave to the audience’s visual and auditory senses and elicited enthusiastic responses, which in turn led to an explosion of interest from media sources, followed by a hologram syndrome across society. This single performance served as a trigger to capture the nation’s attention when complete holograms had received little attention until then.

The first technological feature of Digilog Samulnori was to use sensors that respond to the intensity of sounds generated by percussion instruments and the movements of performers, allowing the holograms to change according to the actions of the performers and the responses of the audience.

As the sound of instruments, for instance, becomes increasingly louder, the dead tree on the stage begins to sprout as flower petals swirl around it. The vibrant movement of the performers is accompanied by butterflies that fill the stage and flutter up into the sky.

The performance is not composed of pre-recorded images, but real-time interaction between digital technology and analogue traditional rhythms, allowing 3D images to change in response to the audience. Normally, 3D movies or broadcasts, for instance, show recordings based on linear scenarios, thereby precluding audience participation.

This is why Digilog Samulnori is referred to as a 4D performance. The stage allows both digital technology and imagination to interact with each other within the same space through physical contact among human beings, thereby turning the stage into an interactive communication venue between audiences, performers, and 3D images.

Specifically, the holographic images and live performers on stage create a time-transcending ensemble, allowing holographic images to instantly respond to the performers’ sounds and movements through technological means, such as sound and motion sensors.

It can be said that this novel venture broke down the boundary between the virtual and the real, creating a new digilog space based on the convergence of art and technology. The second technological feature was that the performance delivered a 3D world without any assistance of specially designed glasses conventionally worn by theater audiences to watch 3D movies such as Avatar. This indicates that the performance applied the same technical mechanism as 3D movies, albeit through a different implementation method. As a result, Digilog Samulnori, characterized as a marvelous performance created through the collaboration of actual performers and virtual holograms, demonstrated that holograms, hailed as the pinnacle of 3D technology, has the potential to transform the performance culture of traditional cultural heritage in the years to come.

2) AI Holographic Content Baekbeom Kim Koo

If Digilog Samulnori features a combination of intangible cultural heritage and hologram technology, the latest creation of “Holographic Baekbeom Kim Koo” aims to connect the holographic projection technology with historical figures in modern and contemporary times.

Kim Koo was one of the leading independence movement activists in Korean history. He was also known by his penname Baekbeom and left a significant legacy on modern Korea. In this project, Baekbeom Kim Koo was recreated using a holographic actor. It was an inspired, ambitious attempt to bring the historical figure back to life, based on thorough research of historical records, with the assistance of state-of-the-art scientific advances such as motion capture and holograms.

A Talk Concert with Baekbeom Kim Koo, which was held in Korea 2019, was met with high acclaim from the audience. It was presented in the format of a history talk show hosted by the interactive hologram, who communicated with the audience as he talked and moved before them.

A wide range of technologies were applied in creating the content of holographic Baekbeom. Interlocking solutions for real-time holographic displays with advanced technologies, such as hologram display technology, facial capture, and motion capture, were used to turn his understudy’s motions and facial expressions into those of the 3D character of Baekbeom.

These technologies enable real-time communication. When the audience ask questions in real time, the AI holographic Baekbeom, in the form of a digital actor, would recreate the ambience of a personal conversation with him. The holographic Baekbeom represented a remarkable advancement in that it involves not just a single technology, but several technologies combined with AI as a central part of the Fourth Industrial Revolution, thereby elevating the application potential of hologram technology to a higher level.

Prospects for Using Holography in Cultural Heritage

Since the performance Digilog Samulnori as the first effort to combine intangible cultural heritage and holography, there has been a greater focus on the potential of hologram technology to be applied to cultural heritage, followed by a cascade of hologram-applied performances and exhibitions.

Holograms are expected to be applied to museum exhibitions in the future. Likewise, although this technology still faces many limitations, the holographic revolution, through steady technological innovations, appears to be developing to the point of blurring the boundary between the physical world and the virtual world.

The cultural industry can adopt hologram technology in a wide array of cultural heritage exhibitions, in addition to intangible cultural heritage performances described above.

This is attributable to holograms’ multi-faceted nature, from two-dimensional and three-dimensional features as floating objects with kaleidoscopic forms and colors, to an interactive four-dimensional world made possible through audience participation. For this reason, holographic technology will likely play a remarkable role in the establishment of digital museums based on tangible or intangible cultural heritage.

As holograms can create a setting identical to an actual exhibition, their use for digital exhibitions holds high potential for application in museums. As the biggest concern facing museums is limitations of offline space, which makes it difficult to display every artifact at once. In this regard, virtual artifacts converted to holographic data are not bound by spatial restrictions, not only for display purposes but also as a way to safeguard cultural properties.

Furthermore, as holographic images of cultural assets can replace actual relics in galleries, there are many more benefits, including comparative exhibitions and research into international cultural properties, preservation of local museums’ regional uniqueness, and restoration of destroyed relics as holographic images, as well as effortless cultural exchanges with foreign countries.