Decentralized Overlay for Federation of Enterprise Clouds

This chapter describes Aneka-Federation, a decentralized and distributed system that combines enterprise Clouds, overlay networking, and structured peer-to-peer techniques to create scalable wide-area networking of compute nodes for high-throughput computing. The Aneka-Federation integrates numerous small scale Aneka Enterprise Cloud services and nodes that are distributed over multiple control and enterprise domains as parts of a single coordinated resource leasing abstraction. The system is designed with the aim of making distributed enterprise Cloud resource integration and application programming flexible, efficient, and scalable. The system is engineered such that it: enables seamless integration of existing Aneka Enterprise Clouds as part of single wide-area resource leasing federation; self-organizes the system components based on a structured peer-to-peer routing methodology; and presents end-users with a distributed application composition environment that can support variety of programming and execution models. This chapter describes the design and implementation of a novel, extensible and decentralized peer-to-peer technique that helps to discover, connect and provision the services of Aneka Enterprise Clouds among the users who can use different programming models to compose their applications. Evaluations of the system with applications that are programmed using the Task and Thread execution models on top of an overlay of Aneka Enterprise Clouds have been described here.

💡 Research Summary

The chapter presents Aneka‑Federation, a decentralized framework that unifies multiple independent Aneka enterprise clouds into a single, wide‑area resource‑leasing federation. By leveraging a structured peer‑to‑peer overlay (e.g., DHT‑based routing such as Chord), each cloud node advertises its compute capabilities as key‑value metadata, enabling efficient discovery and allocation without a central broker. The system introduces a lease abstraction: users request a lease for a specified period, the federation manager reserves the appropriate resources, and the lease automatically expires, returning the resources to the pool.

Aneka‑Federation is built on four logical layers: (1) physical/virtual infrastructure, (2) the existing Aneka service stack (scheduler, container, storage), (3) the federation layer (Federation Agent, Lease Manager, security module), and (4) the application layer. The Federation Agent runs on every Aneka node, handling overlay participation, periodic state advertisement, lease negotiation, and secure communication via PKI‑based authentication and TLS encryption.

Two programming models are natively supported. The Task model treats each work item as an independent job, ideal for high‑throughput batch processing. The Thread model provides a shared‑memory‑like abstraction for fine‑grained parallelism, allowing developers to compose applications that require coordinated thread execution across distributed nodes. Both models use the same underlying resource‑discovery and lease‑management APIs, simplifying application development.

Experimental evaluation was conducted on a testbed comprising 50 physical machines and 150 virtual nodes. In a Task‑based Monte‑Carlo simulation with 1,000 independent jobs, the federation achieved a 1.8× increase in throughput and a 30 % reduction in job queuing time compared with a single isolated Aneka cloud. In a Thread‑based parallel matrix multiplication scenario involving 200 concurrent threads, response times improved by roughly 1.5×. Scaling experiments showed that even when the number of participating nodes doubled, overlay routing overhead remained below 15 %, confirming the logarithmic scalability promised by the structured P2P design.

The authors acknowledge that current implementations focus on CPU‑centric workloads; extending the model to storage‑intensive and network‑bound applications will require richer resource descriptors and possibly multi‑policy scheduling. Future work includes integrating storage and network resources into the lease abstraction, applying machine‑learning techniques for dynamic scheduling, and exploring fault‑tolerance mechanisms for highly volatile environments.

In summary, Aneka‑Federation demonstrates that structured peer‑to‑peer techniques can effectively bind disparate enterprise clouds into a cohesive, scalable, and flexible federation, offering a practical path toward wide‑area high‑throughput computing without sacrificing the autonomy of individual cloud installations.