Research

Project INFO

Collaborators: Jiannan Li, Mauricio Sousa, Jessie Liu, Yan Chen, Ravin Balakrishnan, and Tovi Grossman
Conducted at DGP Lab, University of Toronto
In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI’22)

Abstract

Online synchronous tutoring allows for immediate engagement between instructors and audiences over distance. However, tutoring physical skills remains challenging because current telepresence approaches may not allow for adequate spatial awareness and viewpoint control of the demonstration activities scattered across an entire work area and the instructor’s sufficient awareness of the audience. We present Asteroids, a novel design space for tangible robotic telepresence, to enable workbench-scale physical embodiments of remote people and tangible interactions by the instructor. With Asteroids, the audience can actively control a swarm of mini-telepresence robots, change camera positions, and switch to other robots’ viewpoints. Demonstrators can perceive the audiences’ physical presence while using tangible manipulations to control the audiences’ viewpoints and presentation flow. We conducted an exploratory evaluation for Asteroids with 12 remote participants in a model-making tutorial scenario with an architectural expert demonstrator. Results suggest our unique features benefited participant engagement, sense of presence, and understanding.

System overview

Asteroids is a novel approach for remote physical demonstrations using a swarm of telepresence robots.
A) With Asteroids,remote audience members can inhabit and control small robots on a workbench to follow the instructor’s guidance or roam around looking at activities at various locations and scales.
B) And, a demonstrator can physically interact with the remote audience and use tangible artifacts to control the flow of the demonstration.

Elements of a physical skill demonstration that uses the Asteroids approach.
A) a workspace with multiple (e.g. soldering,assembly, and 3D Printing) zones managed by the demonstrator
B) Asteroids robots
C) Zone tokens
D) Command tokens
E) amember of the remote audience

INTerface design

A) Asteroids’ display shows a videoconferencing interface with the camera feed showing what the audience sees and theaudience members currently inhabiting the robot.
B) the audience web interface displays the camera view of the robot the user currently inhabit, a preview of other robot’s camera view, the map, and directional buttons.

Audience interface

INTeraction design

Using the Map View and the direction buttons on the web interface, a remote user can A) select a robot to watch its camera stream, B) select a target to move the robot, and C) use direction buttons to move the robot. Also, D) when selecting a target near an uninhabited robot, E) the user is automatically transferred to that robot

Asteroids offer five different tangible interactions for instructors to control the flow of the demonstration: A) Demonstrator scan pick up and relocate Asteroids robots. B) Using a red Command token, the demonstrator makes that robot stationary, focusing on a single viewpoint. C) An orange token moves every audience member to that robot. D) The green token indicates a region of interest and makes all robots face that spot. E) Additionally, demonstrators can rearrange the work zones using a pair of zone tokens.

Architecture

Telepresence robot explosion diagram (left), and Asteroids prototype system architecture (right).

Scenarios

A demonstrator notices a change in audience interest and adapt the demonstration flow. A) and B) an Asteroids robot moves away from the soldering iron and moves towards the 3D printer, as the controller of the robot is more interested in the 3D printing process. C) Noticing the movement of this robot, the instructor adds a robot and use a red command token to pin this viewpoint inplace.

An audience member follow the instructor’s work in multiple areas. A) observing soldering in the zone for making individual parts. B) observing robot assembly in the assembling zone.

Fig. 10. The demonstrator construct a workspace tour using the orange command token. A1) The demonstrator places the orange token on a robot pointing at the soldering zone of the workspace. A2) As a result, all audience members transfer to this robot and focus on the soldering zone. B1) The demonstrator continues the tour by placing the orange token on a robot pointing at the assembly zone. B2) all audience members focus on the assembly zone. C1) The demonstrator finishes the tour by placing the orange token on a robot pointing at the 3D printer. C2) all audience members focus on the 3D printer.

An audience member freely explores the workspace. A) The audience member realizes that they already understand the current step as it has been repeated several times. B) The audience member drives the robot to explore other areas of the workspace.C) The audience member studies the finished model in another zone of the workspace.

The demonstrator and the audience member coordinating together to find a good view. A) The robot is moving to go around the branch as the audience member on the robot cannot see the point of cut that the instructor is about to make. B) Noticing the robot want to see behind the branch, the demonstrator rotates the branch towards the robot.

Video

Abstract

Intro

Framework

DATA

SIMulation

design

Video

Data story

Analysis

It is intuitively reasonable to have the view that denser urban areas are more susceptible of becoming epicenters of COVID-19. Because this is what the mass media and government officials tell us. As the hardest hit city in North America, the density of New York City is often highlighted as the one of the major reasons behind its COVID severity. However, based on our analysis, there is no correlation between urban density and covid severity.

Built Density

Built FAR is the relationship between total amount of usable floor area a building has to the land area. For instance, a high-rise condo might have Built FAR of 50, and a bungalow might have Built FAR of 1. And this metric allows us to measure density consistently.

‍

Here's the built FAR map overlaid with case rate, as we can see, the densest built area in New York City such as lower and midtown Manhattan do not have the highest COVID case rate.

‍

Population Density

Since COVID is transmitted through human droplets, we looked at another component population density. Surprisingly, when the base map is changed to population density, there is still no clear correlation between the two.

Our linear regression analysis also validated the initial observations.

Here is a map of education level illustrated by colour, with case rate depicted by the proportional symbol map overlaid on top. A more saturated violet colour indicates higher education attainment, and larger dot size represents higher COVID-19 case rate for each neighbourhood.

Education

As it is shown, the more saturated violet regions have smaller dots, which shows that higher education is potentially correlated with lower case-rates.

In linear regression analysis, education proved to be the strongest correlate of all the variables we explored, by far. As the level of education increases, the case rate decreases. With points tightly scattered about the regression line, and high R-squared of 0.7. This is consistent with the observation from the map.

‍

Income

We also examined the relationship between income per capita versus case rate. From the images below we can see that income and case rate are inversely proportional.

Data sources

New York State (2020). Health Facility General Information. Retrieved 8 July 2020, from https://health.data.ny.gov/Health/Health-Facility-General-Information/vn5v-hh5r

NYC Health (2020). COVID-19: Data. Retrieved 8 July 2020, from https://www1.nyc.gov/site/doh/covid/covid-19-data.page

NYC Planning (2020). PLUTO & MapPLUTO. Retrieved 8 July 2020, from https://www1.nyc.gov/site/planning/data-maps/open-data/dwn-pluto-mappluto.page

United States Census Bureau (2020). American Community Survey Data. Retrieved 8 July 2020, from https://www.census.gov/programs-surveys/acs/data.html

Site Analysis

Pisciculture and floriculture consumption are of vital importance as one fulfills Kolkata’s average residents’ physical need in food and the latter supports their social need in religion practice.

As the core of religious rituals in Kolkata, up to 1,500 tons of fresh-cut flowers are exchanged at Kolkata Flower Market daily before being distributed to retailers and temples. Nonetheless, major flower suppliers travel up to 12 hours to reach Malik Ghat Market, in this primitive transportation process, a large portion of flowers are withered and thus wasted.

Currently, the wetlands directly supply 13,000 tons of fish to Kolkata annually while treating 60% of its sewage water. However, pisciculture is losing its people to other industries as fishermen struggle to make a profit from the shallowing ponds caused by city pollution.

Design Argument

While indigenous ice making has long been a tradition practiced by the Calcuttans. Yet, it is the scarcity of ice which limits the distance and time allowed for fish and flower transportation/storage and constrains the development of by-product industries in Kolkata.

Inspired by the natural sewage treating system and the different levels of water exchanged in the wetland, our proposal challenges the conventional way of ice making by introducing ice of varied qualities, catering to the different needs of environmental cooling, fish/flower transportation and human consumption.

As temporality is a unique aspect of Kolkata, an area prone to monsoons, our project aims at transplanting the agricultural calendar into physical design interventions. By overlaying cycles of pisciculture, floriculture and ice with natural rainfall, solar radiation and temperature patterns, we are introducing three critical points on the hydrological system for further elaboration.

Motivation

Building compelling augmented reality (AR) experiences is challenging due to designers inability to assess how uncontrolled contexts can influence the user experience.
Current approaches include capturing the end user’s environment through imagery or video; and building mini-size studio-interactive spaces to emulate AR environments. However, the former approach can raise privacy concerns and the later approach involves physical setups that are costly and time-consuming.
‍
We propose a non-intrusive, low-cost approach that supports rapid iterations for AR designers through gathering data from crowdworkers in a privacy friendly manner.