Distributed Virtual Supercomputers
Hotel de Santa Fe — Santa Fe, New Mexico
June 24 – 27, 1997
The concept of Distributed Supercomputing (DS) is to interconnect commodity computing building blocks to create multi-Teraflop capability/capacity engines. Sandia is exploring DS as a complement to large-scale vendor machines because we see the development of a “new computing industry” — companies like Intel are moving toward horizontal integration and away from chip-to-supercomputer vertical integration.
Connecting commodity components, potentially from multiple vendors, with commodity interconnects and software poses many challenges and provides many opportunities. The intent of this joint Sandia/Los Alamos/Oakridge/Swiss workshop will be to explore these issues such as:
- Is there a cost advantage to distributed supercomputing?
- What are the implications for software at every level?
- Does DS change the needs at the desktop?
- Can we leverage the academic work on systems like Beowulf?
- Can we turn the abundance of computational capacity we will have in the coming decade into a similar abundance of computational capability?
- What applications can easily use such a system?
- What changes would have to be made to other applications to allow them to use such a system?
Our intent is to keep the workshop informal with lots of room for free flowing discussion.
Agenda
June 25th: Issues, Questions, Challenges
| Time | Event |
|---|---|
| 8:15 – 9:00 | Breakfast and round the room introductions |
| 9:00 – 9:30 | Welcome and meeting purpose. William Camp, SNL |
| 9:30 – 11:30 | Hardware issues in Distributed Supercomputing James Tomkins, SNL Karl-Heinz, LANL Anton Gunzinger, ETH Al Geist, ORNL |
| 11:30 – 1:00 | Lunch |
| 1:00 – 3:00 | Software issues in Distributed Supercomputing Richard Barrett, LANL Pierre Kuonen, ETH Tim Sheehan, ORNL David Greenberg, SNL |
| 3:00 – 3:15 | Break |
| 3:15 – 5:15 | Applications issues in Distributed Supercomputing Ralf Gruber, EPFL Ken Kliewer, ORNL TBD, LANL Robert Clay, SNL |
June 26th: Current Efforts
| Time | Event | |
|---|---|---|
| 8:00 – 8:30 | Breakfast | |
| 8:30 – 10:30 | The computational plant project at SNL Art Hale, Overview Robert Clay, A potential hardware system design Rolf Riesen, A potential system software architecture Pang Chen, Providing balanced, usable I/O Lee Ann Fisk, Supplying a runtime environment David Evensky, Scalable Object Systems for Management and Control | |
| 10:30 – 10:45 | Break | |
| 10:45 – 11:45 | Session A, (Kiva A Room) Al Geist, ORNL, Frontiers of PVM | Session B, (Kiva B Room) Simulating Fluid Dynamics and Turbulence I Michel Deville, EPFL “Parallel Spectral Elements for Complex Fluids” Mark Christon, SNL John Shadid, SNL |
| 11:45 – 1:00 | Lunch, (Hotel Patio) | |
| 1:00 – 2:30 | Session A, (Kiva A Room) Michael Warren, LANL, “Experiences with Loki” Ian Philp, LANL “Obstacles to providing low-latency, high-bandwidth networking in pheterogeneous and wide-area environments” | Session B, (Kiva B Room) Simulating Fluid Dynamics and Turbulence I Open discussion |
| 1:00 – 2:30 | Session C, (Suite 225) Electronic Structure Calculations of Materials Ellen Stechel, SNL Grant Heffelfinger, SNL Alan Wright,SNL Heinrich Roder, LANL Roberto Car, EPFL Dwight Jennison, SNL Jonathan Boettger, LANL Mark Sears, SNL Peter Schultz, SNL Olof Eriksson, LANL Jeff Nelson, SNL Roland Stumpf, SNL Rick Martin, LANL | |
| 2:30 – 3:00 | Break (Kiva A Room) | |
| 3:00 – 4:30 | Session A, continued,
(Kiva A Room) Anton Gunzinger Network for Parallel Commodity Computing Ralf Gruber The Scientific Macintosh | Session B, continued, (Kiva B Room) Adaptive Mesh Refinement Roland Richter, EPFL Unstructured Mesh Adaption Within a True Parallel Environment Mike Wong, SNL Karen Devine, SNL Joe Jung, SNL Walt Witkowski, SNL James Peery, SNL |
| Session C, continued (Suite 255) |
June 27: Preparing for the Future, (Kiva A Room)
| Time | Event |
|---|---|
| 8:00-8:30 | Breakfast |
| 8:30-12:00 | Open discussion of what the key design decisions are that will need to be made, plans to address key bottlenecks, and plans to explore alternatives. Jim Tomkins, Art Hale, Karl-Heinz Winkler, Anton Gunzinger, Ken Kleiwer, Rob Armstrong will lead discussion. |
| 12:00-1:00 | Lunch and wrap up (Kiva B Room) |
Attendees
Swiss Federal Institute of Technology
Roberto Car
Michel Deville
Rolf Gruber
Anton Gunzinger
Pierre Kuonen
Silvio Merazzi
Roland Richter
Marie-Christine Sawley
Sandia National Laboratories
Rob Armstrong
R. Michael Cahoon
William Camp
Robert Clay
David Evensky
David Greenberg
Grant Heffelfinger
Bruce Hendrickson
Rob Leland
John Noe
Tom Pratt
Mark Sears
Jim Tomkins
Michael Vahle
Lucille Verdugo
Pete Wycoff
Oak Ridge National Laboratory
Al Geist
Ken Kliewer
Tim Sheehan
Summary
The Distributed Virtual Supercomputing workshop was held from June 25 – 27, 1997, in Santa Fe, New Mexico. The workshop was coordinated by Sandia in response to their new ASCI project devoted to distributed computing.
People from Sandia, ORNL, LANL, and a couple of Swiss groups participated. Pretty cool deal: build a big time computational machine using commodity parts. Plan for expansion. Plan for widely distributed computing platforms. Below is a partial description of what was discussed.
Some of the participants:
Sandia: Bill Camp, Jim Tomkins, David Greenberg
ORNL: Al Geist, Ken Kliewer, Tim Sheehan
Swiss: Anton Gutzinger (ETH) and Ralf Gruber (EPFL)
LANL: Richard Thomsen, Ian Philp, Mike Warren (Omissions are my oversight. Sorry.)
Sandia and ORNL described the type of applications they are interested in running on such a machine.
Sandia folks described their “Computational Plant” project. The ’97 goal is to build a 100 GFlops peak/60 GFlops achieved machine. Here are some details:
Hardware (presented by Robert Clay)
Tentatively:
- DEC chip (433 or 500MHz)
- Myrinet networking boards. Will explore topologies, but a mesh seems the likely candidate in order to allow for easier growth.
- Flexible partition a la ASCI Red: service, system, compute, I/O.
- 1 I/O node per scalable unit (16 cpus?)
System software (presented by Rolf Riesen)
- Lightweight operating system kernel (initially linux, then puma/cougar port to alpha)
- Portals for fast communication
- Big compiler effort.
- MPI (details below)
Other stuff
Heterogeneity (Al Geist)
Al showed that we have most of the pieces to allow for heterogeneity. They just need to be adapted/adopted/refined. Many presenters/participants shared this view. However, a homogeneous setup will be pursued in the initial phases.
Long distance message passing (Tim Sheehan)
A collaboration between ORNL, Sandia, and the Pittsburg Supercomputing center involves running applications across widely distributed computing resources. With a lot of work, codes (CTH, LSMS) were executed in parallel on machines located at Sandia and ORNL. For details, seehttp://www.lanl.gov/organization/cic/cic8/para-dist-team/DVS/paragon_results.html.
Many argue that this type of computing is simply a political exercise. Regardless, the challenges involved in computing across the country are related to the challenges involving computing between two machines located in the same room. Bill Camp: “The alternative to having two major computing resources located at two sites is to locate them both at one site. Which would you prefer?” Answer: “If they are located at my site, I’m all for the single site model.” (My argument for that site being LANL is that the speed of light is faster at altitude.)
Message passing
Although there were some rumblings regarding other programming models, it appears that explicit message passing is the clear front runner. And probably will be an MPI implementation, although only a subset of MPI’s functionality will get attention with respect to optimization. (That is, adapt a public domain version of MPI, such as mpich or lam.) Hardware folks are quite interested in providing an excellent network infrastructure. In fact, there is talk of not only implementing barriers in hardware, but also reductions/broadcasts/gather/scatter. This could be a big win.
A variety of subgroups will be formed in order to attack the multitude of challenging issues that must be addressed/conquered in order for this project to succeed. It could be argued that these issues are not applicable to us rich folk who can afford $150,000,000 machines, but I would counter that it is exactly these types of issues that must be addressed in order to kick us up to the 10/30/100 TFLOP machines, regardless of cost.
Details regarding how to get in on such discussions should appear shortly. Should be fun…
Richard Barrett
rbarrett@lanl.gov
667-6845
What is it?
“It’s like a tree. It grows, and is pruned to achieve scalability.”
– Jim Tomkins
“You start with a tree. Then you get a forest. Then you have to deal with a continent.”
– Al Geist
“It’s like a cow: we feed it, we milk it, then we slaughter it.”
– I ain’t sayin’…