The xDotGrid Native, Cross-Platform, High-Performance xDFS File Transfer Framework
📝 Abstract
In this paper we introduce and describe the highly concurrent xDFS file transfer protocol and examine its cross-platform and cross-language implementation in native code for both Linux and Windows in 32 or 64-bit multi-core processor architectures. The implemented xDFS protocol based on xDotGrid.NET framework is fully compared with the Globus GridFTP protocol. We finally propose the xDFS protocol as a new paradigm of distributed systems for Internet services, and data-intensive Grid and Cloud applications. Also, we incrementally consider different developmental methods of the optimum file transfer systems, and their advantages and disadvantages. The vision of this paper tries as possible to minimize the overhead concerned with the file transfer protocol itself and to examine optimal software design patterns of that protocol. In all disk-to-disk tests for transferring a 2GB file with or without parallelism, the xDFS throughput at minimum 30% and at most 53% was superior to the GridFTP.
💡 Analysis
In this paper we introduce and describe the highly concurrent xDFS file transfer protocol and examine its cross-platform and cross-language implementation in native code for both Linux and Windows in 32 or 64-bit multi-core processor architectures. The implemented xDFS protocol based on xDotGrid.NET framework is fully compared with the Globus GridFTP protocol. We finally propose the xDFS protocol as a new paradigm of distributed systems for Internet services, and data-intensive Grid and Cloud applications. Also, we incrementally consider different developmental methods of the optimum file transfer systems, and their advantages and disadvantages. The vision of this paper tries as possible to minimize the overhead concerned with the file transfer protocol itself and to examine optimal software design patterns of that protocol. In all disk-to-disk tests for transferring a 2GB file with or without parallelism, the xDFS throughput at minimum 30% and at most 53% was superior to the GridFTP.
📄 Content
The xDotGrid Native, Cross-Platform, High-Performance xDFS File Transfer Framework Alireza Poshtkohi1 , M.B. Ghaznavi-Ghoushchi 1,* 1 Department of Electrical Engineering, Shahed University, Persian Gulf Highway, Tehran 3319118651, Iran Abstract In this paper we introduce and describe the highly concurrent xDFS file transfer protocol and examine its cross-platform and cross-language implementation in native code for both Linux and Windows in 32 or 64-bit multi-core processor architectures. The implemented xDFS protocol based on xDotGrid.NET framework is fully compared with the Globus GridFTP protocol. We finally propose the xDFS protocol as a new paradigm of distributed systems for Internet services, and data-intensive Grid and Cloud applications. Also, we incrementally consider different developmental methods of the optimum file transfer systems, and their advantages and disadvantages. The vision of this paper tries as possible to minimize the overhead concerned with the file transfer protocol itself and to examine optimal software design patterns of that protocol. In all disk-to-disk tests for transferring a 2GB file with or without parallelism, the xDFS throughput at minimum 30% and at most 53% was superior to the GridFTP. Keywords: xDFS Protocol, DotDFS Protocol, xDotGrid Platform, High Throughput File Transfer, Highly Concurrent Systems, Event-Driven Architecture, Grid Computing, Cloud Computing, Internet Services, .NET Framework
- Introduction As a fundamental result of information age, communication mechanisms and data transmission protocols provide an infrastructural foundation for the emergence and evolution of enormous computing paradigms, and integrating data access on a world-wide scale. The two open standards protocols of HTTP [1] and FTP [2] have provided basic file transfer functionalities. To overcome the problems concerned with these two protocols that are mainly due to the overheads of the TCP protocol in its window-based congestion control mechanisms used, the GridFTP protocol [3] [4] [5] has been proposed. In [6] [7] [8] [9] [10] [11] [12], we introduced a hybrid concurrent file transfer protocol, called as DotDFS, integrated with a set of event-driven and threaded-based models. DotDFS was the first file transfer protocol that, in addition to propose a new computing paradigm in the field of data transmission protocols, unveiled many architectural problems regarding the FTP and GridFTP protocols. The TCP protocol has been used as a transport-level communication protocol on the Internet over the years. However, TCP is a rather old communication protocol designed in the 1970s. Many problems regarding the TCP have been reported such as its debility to support the increasing speeds of modern networks. One commonly-used way to reduce the overheads posed by TCP is to simultaneously choose an optimum number of TCP connections and the TCP socket buffer size, which are discussed fully in [6] [7] [9]. This paper tries as possible to minimize the overheads concerned with the file transfer protocol itself and to examine optimal software design patterns of that protocol. This goal plays a key role to reduce the problems associated with TCP overheads which decrease the throughput of the file transfer system. It increases dramatically the entire system efficiency and reliefs exposed drawbacks. However, it is necessary noting that the XDSI structure (refer to the Sections 3.1 and 3.2) allows xDFS protocol to operate over more optimum transport protocols (e.g., SCTP [21]) than TCP. XDSI and the xDFS framework will bring various opportunities together for research communities to implement non-TCP XDSI-enabled drivers for the sake of achieving a virtually zero-percent-overhead file transfer system currently in userspace. xDotGrid project [6] is a new effort to establish an OS-based framework for rapid development of high- performance distributed paradigms heavily relied on the .NET ECMA standards [13] [14] [15]. The basic infrastructure is inspired from the mostly concepts available in Cluster, Grid and Cloud environments. From the viewpoint of the developer users, the xDotGrid.NET framework provides a set of rich, cross-platform, object-
- Corresponding author. E-mail addresses: alireza.poshtkohi@gmail.com (A. Poshtkohi), ghaznavi@shahed.ac.ir (M.B. Ghaznavi-Ghoushchi).
A. Poshtkohi, M.B. Ghaznavi-Ghoushchi
oriented, high-performance and low-level C++ class libraries which enable quick and solid development of the next- generation networked/distributed applications and paradigms. From the viewpoint of the xDotGrid core, it supplies some critical services developed with parts in kernel mode and some other parts in user space. What will be presented in this paper is not just the port of an existing code to the native code. xDFS has been designed from scratch, and this paper explains the author
This content is AI-processed based on ArXiv data.