Multimodal Family History & Digital Heirlooms

This is an ongoing exploration of memory-keeping in my own family: combining 3D scans, photos, audio recordings, recipes, gps locations, and screenshots. This research is concerned with multimodality in the media that is preserved, as well as the way it is displayed. I hope to design a usable and compelling solution for family archival that can accommodate both analog and digital memories, so that other families can use it and eventually extend this into community archives for farms, organizations, and more.

Part of this research will take place from April-May in Berlin, at ZK/U, as part of a two-month residency program. I am researching the capabilities of server hosting, machine learning, data visualization, immersive technology, and generative art to weave the ephemeral bits of history held by my family. I hope to design an organizational structure around the goal of celebrating and inspiring nostalgic gaze rather than creating a highly indexed structure that functions as evidence.

👉 How can memory be captured and revisited multimodally? Nonlinearly?

👉 What is the role of archival amidst techno-optimism? Platform capitalism? Byt rot?

👉 How can we collect in a way that expands our ability to reflect in the future?

👉 Can our family archives reflect the nostalgic gaze of the people who created its contents?

Photo

3D Objects/Environments

Screenshots/Born-Digital Media

I am researching how the following outputs could sew together digital and analog heirlooms: these are ways that memory can be experienced immersively and nonlinearly. My research is structured around the tender and expansive compilation of memory. Each of these outputs represent one chapter of my research, each of which I am seeking funding

Video

GPS/Mapping

Smell

Sound

Document

Touch

Modalities of Memory

Expand Modalities of Context

With technology being woven into our lives, memories are being created through myriad modalities. Context is offered not just through word, but location, smell, touch. This research will explore how these ephemeral bits of memory can be woven together.

Design Challenges

Tech Sovereignty

This project aims to decenter corporate ownership of our memories and personhood. I want to create something that allows the most important artifacts of our family history to be gathered and stored outside of services like Google Photos or AWS. Memory should not be confined by corporate interests and copyright constraints.

Nonlinear Storytelling

A core goal of this project is to use multimodality to embrace nonlinear storytelling, I am researching ways to add dimension to the practice of oral history through server hosting, data visualization (e.g. Soot), 3D environments (e.g. Unity or New Art City), nonlinear storytelling tools (e.g. Twine), and gaming scripting languages (e.g. Ink). I’m curious about ways to experience a family history, and how data visualization or immersive storytelling can create serendipitous discovery.

Longevity

Being that this project involves creating a shared digital repository for a family’s heirlooms (particularly those that are born-digital), I am researching the best ways to maintain the longevity of data. This includes utilizing reliable cloud storage options alongside NAS, while implementing RAID (Redundant Array of Inexpensive Drives) to ensure security and longevity.

Technological Accessibility

While I’m building a family history repository for my own family, my goal is to offer the structure I design as a resource for other families. In order for this structure to repeatable, I’m weighing the benefits and challenges of a more beginner-friendly cloud-storage solution (e.g. Nextcloud) vs a more autonomous but more involved open-source server framework (e.g. TrueNAS).

Multigenerational Organizational Framework

With each generation, family history becomes more and more multi-pronged. As heirlooms age alongside the family, they gain more context and different backstories. A multimodal family history framework must take this into consideration through user-management, as well as structure of file management. The organizational structure of an entire family’s history is a core focus of my design research. I am looking to expand on traditional Chinese family genealogies (家谱), which are traditionally patriarchally organized, as well as as take inspiration from other ancient ancestral practices of record-keeping.

File System

Nonlinear Storytelling (using 3D environments, OBS, and game engines to create a remembrance experience)

Visualized Map of File System (this is Soot, a new filesystem visualization tool)

Sculpture (researching touch and smell as expressions of memory and integrating them into form)