Spring 2020

Master’s Graduate Thesis

The student final project is undertaken in the last two semesters of the program, and is in part a culmination of graduate study that synthesizes previous coursework and investigations. Students are responsible for defining the area of investigation, for the research that informs their investigation, for organizing the study and its methods, and for documenting the results.

User Experience
Augmented Reality
Speculative Design
Data Sense

Full Document ︎
Pandemic Symposium ︎

RESEARCH 

Abstract

Smart Cities are characterized by information flows among humans and ubiquitous sensing technologies known as the internet of things (IoT). These sensors ere embedded in mobile devices and the built environment. While IoT’s affordances aim to make life easier for citizens, its invisibility, and reliance on human data raises issues of transparency and ethics for the future of IoT design.

Designing for transparency is difficult when sensors and data are hard for users to conceptualize (Lupton, 2017). Information flows are dynamic and volatile, and are fragmented and translated across physical-digital spaces. The idea of locating, finding, and interacting with sensors and data in situ could perhaps render them transparent in ways that acknowledge their spatial qualities to help bridge the digital–physical divide. Below is my primary research question followed by three subquestions:

How can the design of a mixed reality experience facilitate transparency of IoT sensors and data through citizen engagement with its infrastructure?

(Sq. 1) Engagement
How can the design of multimodal interventions present initial touch points for the user to access the IoT infrastructure over time and space?

(Sq. 2) Attunement
How can the design of contextual layers of information translate the IoT infrastructure’s functionality and intent?

(Sq. 3) Expansion
How can the design of applied interactions facilitate an embodied awareness of information while promoting user agency?

Methods

Theoretical Frameworks

This research is informed by several frameworks:

Multisensory Acts of Noticing helps humans gain insight into how systems function outside of our anthropocentric norm. Stages of noticing include three tactics: engagement, attunement, and expansion.

The DASS framework gives a structured and holistic view of how designers can implement awareness of information into the design of interactive and shared systems.

Smart citizenship is derived from literature covering aspects of citizen engagement. Smart citizenship does not refer to the cognitive capabilities of citizens; rather, it is a broader concept that encompasess an ethos of citizen agency through grassroots, bottom-up participatory design.

Data sense involves the entanglements of human senses with digital sensors in the act of sensemaking. This investigation is framed from the perspective of data sensemaking. It recognizes materiality of data as a lively thing distributed spatially and contextually in the digital–physical milieu of sensory embedded environment.

Precedent Analysis 

As a part of this investigation, I researched existing design applications, experiences, and frameworks that center around citizen sensing, IoT awareness, and augmented reality. Collectively, these examples advocate for citizen’s right-to-know and uniquely address the concept of transparency and smart citizenship through place-based sensing, access to open source data, and active participation.

What I took from these precedents:

• Citizens as sensors—citizens understand sensory environments through participatory acts of sensing (Gabrys, 2018).
  
• Sensing kits are placed based and require mobility to access. Walking is a part of understanding the spatial scope of embedded IoT devices.
  
• Transparency is achieved through the use of open source and open data.
  
• Data is generated and collected through the willingness and active use by a participant.
  
• Sensor activity can be made evident through public interactive displays that convey ambient information.
  
• Mobile devices can inform users of local data collection.
• ARIoT can be experienced in natural ways that use proximity-based sensing and BLE beacons to notify of the user of a sensor near by.
  
• Sensing nodes do not always take the form of mobile devices. Some are rendered as eggs and other playful forms.

STUDIES

Studies are organized in the form of three subquestions that focus on different aspects of Designing for Awareness. Each subquestion addresses a tactic of noticing derived from the primary framework, The Acts of Noticing: Engagement, Attunement, and Expansion (Liu et al., 2018).

Engagement is the shared physical experience of the environment.
Attunement is the ability to sense the livelihoods of nonhuman collaborators.
Expansion explores the blurring of the human–technological division. 

Persona + Senario

This investigation follows the persona, AVA, a twenty-five year old college student and **unaware pedestrian as she commute to class over a course of time. While she has heard of the term “smart” she does not know if she lives in a smart city or not. She has grown up with digital technologies and rarely thinks about their mehanics. She fears and is convinced her phone listens to her speak sometimes and wonders about how the cloud works—but does not obsess over those unknowns.

The context is Hillsborough Street, Raleigh, NC, a typical of a college street in a medium–sized city. It is active and lively, populated with a variety of business, restaurants, coffee shops, bars, and pedestrian activity. On one end of the street is a traffic circle characterized by constant, transient motion. Embedded within this physical landscape are thousands of IoT sensors—sensors that are aiding Ava's journey (unbeknownst to her) as she walks to and from class. 

**The concept of “unaware pedestrian” is supported by literature on solitary mobility in the city, the anesthetization of users to digital information when moving through public space, and the ambient commons, (Bull, 2004; McCullough, 2015). Modern pedestrian mobility is oftentimes solitary with user attention directed towards their private worlds. In my observations, many pedestrians were alone with headphones on—heads down looking at content and not at the environment itself. A function of this investigation is to ask how design can snap a pedestrian out of that solitary state into one of awarenesses of the invisible IoT infrastructure.

Study 1:

Engagement
How can the design of multimodal interventions present initial touch points for the user to access the IoT infrastructure over time and space?

This study focuses on eliciting engagement through initial contact with the design system. It takes a step back and asks how potential technological interventions can bring the pedestrian into Awareness by engaging more fully with sensors and data recognized in the AR application. When paired with smart citizen values, study one explores the idea of citizen right-to-know and accessibility of IoT information. As such, this study investigates a range of possibilities for an unaware pedestrian to gain access to IoT information in situ.

Findings propose an IoT sensor can be transparent through the design of multimodal, technological touchpoints in public space. These touchpoints vary in visibility, modality, location, and the type of information they provide. The result is a concept map that synthesizes these variables to deduce, where and when, these touchpoints should intervene.  

These frictions range from evoking ambient, thinner interactions that are successful in provoking a sensory awareness of space, to explicit, thicker interactions that enable access to detailed information. They explore the effects of facilitating awareness and in indirect proximity to an IoT sensor. The former alters pedestrian routes while the latter engages the user with the sensor in situ.

Study 2:

Attunement
How can the design of contextual layers of information translate the IoT infrastructure’s functionality and intent?

This study explores contextual and expressive awareness of IoT sensors and data. It details how they can bring awareness to themselves in the environment through tactics that highlight, locate, describe, or simulate their mechanics and intent. It then explores the format in which the information can be given: overlaid text, hyperlinking to websites, providing detailed infographics. It also questions how sensors and data can be differentiated from each other and convey information that is aligned with their purpose—what they do, how they do it, how sensitive a sensor may be to a user.

This study proposes a data schema that can be applied in the design of data (and sensors). Metadata can be represented by tagging, nesting, or overlaying information on some form—this information can shift in hierarchy according to the user’s preferences. 

Study 3:

Expansion
How can the design of applied interactions facilitate an embodied awareness of information while promoting user agency?

Expansion covers interactions that are synonymous with embodiment in the context of AR. Embodied Awareness unfolds through the experience of walking, locating, accessing, and interacting with sensors and data in augmented space. It is a multisensory, physical experience that can be facilitated in design through interventions that initiate movement (such as leaving traces for a user to follow), provoking first person perspective—the user is the enactor rather than the observer—and through blending in with the environment (Hummels & Dijk, 2015).

This study proposes that interactions are synonymous with embodied sensemaking in AR. As such, designers should prototype for embodied experiences as well as cognitive ones.

Embodied experiences can facilitate a spatial and material understanding of IoT infrastructure through the placement of assets and applied interactions. A cluster of assets surrounding the user can provoke a sense of immersion, while distant and unreachable assets can initiate mobility and re-orientation. It discusses the downsides of embodied interactions such as cognitive and perceptual hindrances caused by distortion and resolution, viewpoint mismatching, and illegible displays of information (Olshannikova et al., 2015).

DISCUSSION

Scenario Map + Framework

As a whole, the studies comprise a system in which an unaware pedestrian, Ava can come into contact with and engage with invisible IoT infrastructure. This sytem relies on touchpoints in the phyical environment (subquestion 1) and embodied interactions in AR (Subquestion 2 and 3). The scenario map outlines her user journey. The sketch framework diagrams the system in its entirety. 

Design Principles

Findings from these studies suggest principles to apply when designing for awareness or in the data and IoT domain:

Designing for embodied sense making acknowledges the material, spatial, and embedded nature of digital information that can help bridge the digital-physical divide.

Leveraging IoT infrastructure agency through metaphor and enchantment. It is clear from substantial research that embedded sensors and data have a degree of agency and influence on human lives. And the way we render these things in the design of smart devices affects human perception and ways of relating to them. Enchantement can provoke awareness in ways that are not necessarily cute, or anthropomorphized—but them as agents in their own right.

By remaining invisible, discrete, and abstract, there will continue to be an imbalance of power between users and technological artifacts. Using More than Human theories as guiding frameworks can broaden solution possibilities in the design of complex systems—especially in the ideation process by approaching systems through alternative perspectives. Findings can then be framed from a user-centered perspective to make designs more viable and usable.

While the scope of this project is largely speculative, its content is of ethical concern for the future of IoT design. Should IoT infrastructure remain seamlessly invisible or provoke awareness through added friction? Does citizen right–to–know mean that users must seek out information themselves or that transparency is made evident through design? Maybe this issue will not be as pressing to the next generation of digital natives who are so embedded in digital infrastructure their cares and concerns are different from ours. But, I’d argue that it should be regarded as essential as usability testing in the design of IoT spaces. Citizens should have the choice to engage, care, or delete. Designing for awareness can facilitate just that.