In a technological landscape that is quickly moving toward dense multi-CPU and multi-core computer systems, where using multithreading is an increasingly popular application design decision, it is important to choose a proper model for distributing tasks across multiple threads that will result in the best efficiency for the application and the system as a whole. The work described in this paper creates, implements and evaluates various models of distributing tasks to CPU threads and investigates their characteristics for use in modern high-performance network servers. The results presented here comprise a roadmap of models for building multithreaded server applications for modern server hardware and Unix-like operating systems.
odern server applications cannot avoid taking note of recent developments in computer architecture that resulted in wide-spread use of multi-core CPUs even on low-end equipment [1]. Different equipment manufacturers use different approaches [2][3][4] with the same end result of having multiple hardware processing units available to the software. This is not a recent development [5,17] but it has recently received a spotlight treatment because of the proliferation of multi-core CPUs in all environments. This has resulted in changed requirements for high-performance server software.
The prevailing and recommended model for building high-performance network servers for a long time was based on asynchronous IO and event-driven architectures in various forms [6][7][8]. Shortcomings of this model grew as the number of CPU cores available in recent servers increased. Despite the complexities implicit in creating multithreaded programs, it was inevitably accepted as it allows fuller use of hardware possibilities [9], though recently the pure multithreaded model is combined with the multiprocessing model for increased performance or convenience [10].
This work studies the different ways tasks in a databaselike network server can be assigned to CPU threads. To investigate the various models that can be used in this mapping, we have used a flexible memory database server This work is supported in part by the Croatian Ministry of Science, Education and Sports, under the research project “Software Engineering in Ubiquitous Computing”. I. Voras and M. Žagar are with Faculty of Electrical Engineering and Computing, University of Zagreb, 10000 Zagreb, Croatia. (e-mail: ivan.voras@fer.hr, mario.zagar@fer.hr).
(described in [11]) which is intentionally created to be capable of various models of distribution of workload tasks to CPU threads. It allows us to experiment and measure performance characteristics of these models in directly comparable circumstances, using benchmarking clients made as a part of the referenced project.
Our conclusions are that there are significant differences between the multithreading models, which are visible both in the performance curve and the best performance each model can achieve.
This paper describes major multithreading models applied in practice and described in literature in the second section. It continues with the description of the specific implementation of a memory cache database project which can employ any of several multithreading models and the protocol of the evaluation in section three. The main part of the paper is comprised of evaluation results in section four, after which follows the conclusion in section five.
In this work the term “multithreading model” refers to a distinct way tasks in IO (input-output) driven network servers are distributed to CPU threads. It concerns itself with general notion of threads dedicated for certain class of tasks, rather than the quantity of threads or the tasks present at any one point (except for some special or degenerate cases like 0 or 1).
We observe that in a typical network server there are three classes of tasks that allow themselves to be parallelized or delegated to different CPU threads: Using these three classes of tasks, there are several models recognized in practice and literature [12]. At one part of the spectrum is the single process event driven (SPED) model, where all three classes of tasks are always performed by a single thread (or process), using kernel-provided event notification mechanisms to manage multiple clients at the same time, but performing each individual task sequentially. On the opposite side of SPED is staged eventdriven architecture (SEDA), where every task class is implemented as a thread or a thread pool and a response to a request made to the client might involve multiple Ivan Voras, Mario Žagar
Characteristics of multithreading models for high-performance IO driven network applications different threads. In our implementation SEDA is implemented as the connection accepting thread with also runs the tasks of the network IO threads (or simply “network threads”) and payload threads (or “worker threads”). Between the extremes are the asymmetric multiprocess event-driven (AMPED) model where the network operations are processed in a way similar to SPED (i.e. in a single thread) while the payload work is delegated to separate threads to avoid running long-term operations directly in the network loop and symmetric multi-process event driven (SYMPED) which uses multiple SPED-like threads, each processing several client connections. A special class of AMPED is a thread-per-connection model (usually called simply the multithreaded or multiprocessing -MT or MP -model) where connection acceptance and garbage collecting is implemented in a single thread which delegates both the network communication and payload work to a single thread for each connected client (this model will not be specially investigat
This content is AI-processed based on open access ArXiv data.