Data Tsunami: BGI Transfers Genomic Data Across Pacific at Nearly 10 Gigabits/Second
By Kevin Davies
June 29, 2012 | BGI, the Chinese genomics organization based in Shenzhen and Hong Kong, has transferred genomic data across the Pacific Ocean at a sustained rate of almost 10 Gigabits per second (Gbps).
“The massive speeds demonstrated are, to our knowledge, the fastest file transfers over public WAN at such distances using commodity hardware,” said Michelle Munson, CEO of Aspera, the company that provided the key file transfer software, “and make possible a host of new data intensive applications in genomics analysis and medicine in which geography is truly no limit.”
The transfer took place on June 22, as part of an event in Beijing to celebrate a new 10-Gigabit US-China network connection, supported by groups including Internet2, the National Science Foundation (NSF), and Indiana University. It was conducted over a new link connecting US and Chinese research and education networks.
During the demonstration, BGI transferred 24 Gb genomic data from Beijing to UC Davis in less than 30 seconds. (A file of the same size sent over the public Internet a few days earlier took more than 26 hours.) The measured data rate is equivalent to moving more than 100 million megabytes -- over 5,400 full Blu-ray discs -- in a single day.
Joining BGI’s Xing Xu (director of cloud computing) in the demonstration were Dawei Lin, director of the UC Davis Genome Center Bioinformatics Core/NCBI; and Xingzhe Fan, chief scientist at Aspera, the California software company that provided the data transfer file protocol.
“The 10-Gigabit network connection is even faster than transferring data to most local hard drives,” said UC Davis’ Lin. (Earlier this year, BGI and UC Davis announced the creation of a genome center, BGI@UCDavis, on the UC Davis Sacramento campus.)
Lin went onto say that the use of a 10-Gigabit connection will be “groundbreaking,” allowing genomics researchers to transfer data rapidly, “and bring the best minds together to better explore the mysteries of life science.”
“Aspera is thrilled to have shown what is possible in terms of maximum speed large data transfer over the open Internet,” said Munson in a statement to Bio-IT World.
According to BGI’s Xing Xu, this marks the first time that large genomic data were transferred between China and the US over a 10-Gigabit network. BGI “looks forward to the potential opportunity to incorporate this breakthrough into our service capabilities to facilitate more rapid and efficient exchange of big genomic data globally.”
"I am very glad that BGI had the opportunity to perform the demo, and tell a story about how the fast speed connection may change the picture of genomic data exchange completely," Xu told Bio-IT World. "It is an astounding speed for data transfer across two continents at a record of 8 Gbit/s through a 10 Gbit/s connection. It's the combination of the right software, right hardware, right technology and right time."
BGI says the 10-Gigabit Internet connection may provide a significant new tool for tackling big data and increasing scientific collaboration, education and cultural exchange between China and the US. Xu acknowledges that to offer the same speed for routine transmission will be more challenging. Transfer speeds speed will be constrained not only by the network load but also by economical issues.