Opportunistic Content Search of Smartphone Photos

📝 Abstract

Photos taken by smartphone users can accidentally contain content that is timely and valuable to others, often in real-time. We report the system design and evaluation of a distributed search system, Theia, for crowd-sourced real-time content search of smartphone photos. Because smartphones are resource-constrained, Theia incorporates two key innovations to control search cost and improve search efficiency. Incremental Search expands search scope incrementally and exploits user feedback. Partitioned Search leverages the cloud to reduce the energy consumption of search in smartphones. Through user studies, measurement studies, and field studies, we show that Theia reduces the cost per relevant photo by an average of 59%. It reduces the energy consumption of search by up to 55% and 81% compared to alternative strategies of executing entirely locally or entirely in the cloud. Search results from smartphones are obtained in seconds. Our experiments also suggest approaches to further improve these results.

💡 Analysis

Photos taken by smartphone users can accidentally contain content that is timely and valuable to others, often in real-time. We report the system design and evaluation of a distributed search system, Theia, for crowd-sourced real-time content search of smartphone photos. Because smartphones are resource-constrained, Theia incorporates two key innovations to control search cost and improve search efficiency. Incremental Search expands search scope incrementally and exploits user feedback. Partitioned Search leverages the cloud to reduce the energy consumption of search in smartphones. Through user studies, measurement studies, and field studies, we show that Theia reduces the cost per relevant photo by an average of 59%. It reduces the energy consumption of search by up to 55% and 81% compared to alternative strategies of executing entirely locally or entirely in the cloud. Search results from smartphones are obtained in seconds. Our experiments also suggest approaches to further improve these results.

📄 Content

Figure 1: Theft caught in the background of a family photo (Source: CNN [2]). Although this particular photo was not taken with a smartphone, it exemplifies the opportunistic value of photos taken by others Opportunistic Content Search of Smartphone Photos

Technical Report TR0627-2011, Rice University

Ardalan Amiri Sani *, Wolfgang Richter §, Xuan Bao †, Trevor Narayan †, Mahadev Satyanarayanan §, Lin Zhong *, Romit Roy Choudhury †

• Rice University, § Carnegie Mellon University, † Duke University

ABSTRACT Photos taken by smartphone users can accidentally contain content that is timely and valuable to others, often in real-time. We report the system design and evaluation of a distributed search system, Theia, for crowd-sourced real-time content search of smartphone photos. Because smartphones are resource-constrained, Theia incorporates two key innovations to control search cost and im- prove search efficiency. Incremental Search expands search scope incrementally and exploits user feedback. Partitioned Search lev- erages the cloud to reduce the energy consumption of search in smartphones. Through user studies, measurement studies, and field studies, we show that Theia reduces the cost per relevant photo by an average of 59%. It reduces the energy consumption of search by up to 55% and 81% compared to alternative strategies of executing entirely locally or entirely in the cloud. Search results from smartphones are obtained in seconds. Our experiments also suggest approaches to further improve these results.

Author Keywords Crowd-sourced photos, mobile systems, energy efficiency.

1. Introduction Modern smartphones allow us to take photos on the go, capturing whatever we find interesting. We do selectively share some of them with friends and even the public, e.g., through social network websites such as Facebook and Flickr. However, the majority of smartphone photos will not be shared, or possibly even transferred to another computer. Our work was motivated by many important scenarios in which photos captured by a smartphone user become vitally important to others, often in real-time. For example, when a child is lost during a holiday parade, photos by smartphone users nearby become very valuable to the police and parents [1]. As another example, a family photo may reveal a theft [2] (see Figure 1). As yet another example, a sports reporter would like to find the smartphone photos taken from the best angle at the time of a goal during a soccer game. The key question is: How can an interested party find relevant smartphone photos, in real-time? Relevance of a photo is not only determined by the metadata of the photo (e.g., time and location), but also by its content (e.g., “a girl with a red coat”). Our answer to this question is a distributed search service called Theia. Theia considers registered smartphones as distributed data- bases and allows a third party to compose a query and pushes it into these smartphones to find out photos that match the query. The query is a piece of code that examines not only the metadata but also the content of a photo. We focus on architecture and sys- tem design of Theia here, deferring issues such as incentive mech- anisms and privacy control for future. In particular, we focus on how Theia helps its users control search cost and improve search efficiency. Unlike existing search systems whose databases are hosted by powerful data centers, Theia’s databases are hosted by resource-constrained smartphones. Executing a query inside a smartphone can be resource-intensive and incur high cost to the smartphone owner that will eventually be paid by the search user. In view of the large number of smartphones Theia may search, the cost to the search user can be significant. Theia incorporates two key innovations toward solving the above problem. Incremental search allows the search user to submit a cost budget along with a query and Theia will limit the search scope according to the budget. It tracks which photos have been searched by the query and allows the search user to effectively expand the scope by submitting the query again with a new budg- et. As in any search system, a search result, or a matched photo, is not necessarily what the search user is looking for or relevant. The objective of the incremental search is to help a search user find relevant photos with lowest cost per relevant photo. Partitioned search leverages the cloud to reduce the execution energy cost of a query in a smartphone. Based on the selectivity and energy cost of the predicates in the query and the wireless energy cost of offload- ing a photo, Theia dynamically identifies the predicates to be evaluated in the cloud and selectively offloads photos to reduce the energy cost of the smartphone.

2

Figure 2: Architecture of Theia and information flow be- tween its components

Figure 3: An example Theia

This content is AI-processed based on ArXiv data.