Cost reduction is one of the important benefits of cloud computing. One way to achieve this benefit is through resource consolidation and dynamic provisioning among applications to enhance resource utilization. Another method is to reduce the cost for the construction and operation of data centers. The construction and intelligent operation of very large data centers with hundreds of thousands of servers (mega data centers) at low-cost locations can facilitate both methods, and thus become very attractive to cloud providers. For example, Microsoft is building a data center in Chicago that will potentially pack in up to 300,000 servers.

One big challenge in mega data centers is the problem of scalability. Due to the large number of servers and high volume of intra- and inter-data center traffic, traditional tree-based interconnecting architecture can no longer meet the requirements of mega data centers with many servers. Also, it cannot guarantee the bandwidth requirement between any pair of servers in the data center. Many research projects have attempted to address the problem. One thread of research focuses on new architectures based on traditional switch routing, such as the VL2 and Fat-free networks. Another thread of research proposes to build server-centric networks, in which the servers are also involved in packet forwarding.

This paper proposes DPillar, a server-centric data center network architecture, that is closely related to the classic wrapped butterfly network. DPillar is a symmetric network with no bottleneck, which can easily scale to millions of servers. It is very efficient in terms of both computational overhead and packet routing. In order to ensure high bandwidth between any pair of servers, a multipath routing scheme is utilized in DPillar. In addition, DPillar also offers very good fault tolerance to server and switch failures.