Processing sensory data close to the data source, often involving Edge devices, promises low latency for pervasive applications, like smart cities. This commonly involves a multitude of processing services, executed with limited resources; this setup faces three problems: first, the application demand and the resource availability fluctuate, so the service execution must scale dynamically to sustain processing requirements (e.g., latency); second, each service permits different actions to adjust its operation, so they require individual scaling policies; third, without a higher-level mediator, services would cannibalize any resources of services co-located on the same device. This demo first presents a platform for context-aware autoscaling of stream processing services that allows developers to monitor and adjust the service execution across multiple service-specific parameters. We then connect a scaling agent to these interfaces that gradually builds an understanding of the processing environment by exploring each service's action space; the agent then optimizes the service execution according to this knowledge. Participants can revisit the demo contents as video summary and introductory poster, or build a custom agent by extending the artifact repository.
Sensory data is used for fueling and optimizing pervasive applications, from autonomous driving [1] to smart cities [2]. This is supported by the growing computational power of embedded devices and Edge servers that support low-latency processing close to the data source. The precise requirements how this processing must be done are specified through Service Level Objectives (SLOs); real-time applications-like point cloud mapping [3]-might specify a maximum target latency. Yet, resources on Edge servers are limited, whereas client demand is fluctuating; this inevitably leads to situations where resources do not suffice to satisfy SLOs across multiple competing clients and applications. To ensure SLO fulfillment, autoscaling solutions-like Kubernetes [4]-have specialized in adjusting applications according to varying demand; however, the default mechanism is provisioning additional resources. Also, it cannot be assumed that computation can be offloaded to nearby devices [5]. As the context changes dynamically and services cannot rely on predefined mechanisms (e.g., offloading or resource scaling), the processing services must autonomously find actions that optimize their SLO fulfillment.
To facilitate the transition to flexible and context-aware autoscaling of processing services, we developed a two-fold approach [6]: MUDAP-our Multi-Dimensional Autoscaling Platform-supports fine-grained adjustments of the service execution and the allocated resources; notably, this permits dynamic adjustments to service-specific parameters, like the size of Machine Learning (ML) models or input tensors. Second, we presented RASK, a scaling agent that uses Regression Analysis of Structural Knowledge to interpret the effect of different parameter assignments on the SLO fulfillment, and then infer optimal scaling actions. Together, they enable flexible processing services that scale different parameters according to the context-a behavior called multi-dimensional elasticity [7]. Thus, services can trade off less critical aspects (e.g., client experience) to sustain critical SLOs (e.g., latency).
This demo first introduces the architecture of MUDAP and RASK; next, we design a scaling agent that uses these interfaces to optimize the performance of three stream processing services, co-located on an Edge device. To provide insights into this operation, we visualize the agent’s understanding of the processing environment and show how its internal model and the SLO fulfillment improve parallel. We complement this with an introductory poster [8] for quickly conveying the high-level idea; additionally, we host the demo application at a public URL, provide a video summary [9] to it, and share an artifact repository [10] for revisiting the demo contents.
In the following, we present an architecture for contextaware autoscaling of stream processing services, involving two components: MUDAP and RASK. MUDAP exposes service-specific parameters for fine-grained adjustments of the processing environment, while RASK uses the interfaces for interpreting and optimizing the environment. Later, we visualize the RASK agent’s internal models and show how increasingly accurate world models improve decision-making.
The MUDAP platform is introduced in Figure 1 in four steps: 1 ⃝ It streams and buffers sensory data (e.g., video frames) at a nearby device, where multiple containerized processing services run. 2 ⃝ The data is processed, e.g., by running video inference. 3 ⃝ It continuously exports processing metrics to a time-series DB; this includes metrics about service executions (e.g., latency or data quality) and the associated resources (e.g., CPU limit). These variables describe a service’s state space; those variable that can be directly adjusted Fig. 1: Architecture of the MUDAP platform [6]: sensor data is 1⃝ buffered and 2 ⃝ processed by containerized services; 3 ⃝ service and container states (i.e., processing metrics) are collected in a time-series DB. Lastly, 4 ⃝ a scaling agent interprets these states, develops a policy, and adjusts service configurations and their containers through a REST API. ⃝ create a tabular structure from time-series data and train regression functions; 2 ⃝ supply functions, SLOs, and parameter bounds to numerical solver; 3 ⃝ optimize parameter assignments for all monitored services and adjust values through MUDAP API.
form the action space. For example, video resolution (i.e., data quality) can be scaled dynamically. To invoke actions for a service (e.g., change its data quality) we offer a REST API in the container. 4 ⃝ It optimizes service execution by coupling an agent to these interfaces. This allows arbitrary implementations of autoscalers; in our case, the RASK agent.
To optimize the execution of pervasive stream processing services, we present RASK alongside Figure 2 in three steps: 1 ⃝ The agent models the behavior of the processing environment by fitting regression functions, using tabular metr
This content is AI-processed based on open access ArXiv data.