Poor-man's super-(mini?)-computer revisited in 2011? Advice?

Report or discuss software problems and other woes

Moderators: arango, robertson

Post Reply
Message
Author
jpringle
Posts: 108
Joined: Sun Jul 27, 2003 6:49 pm
Location: UNH, USA

Poor-man's super-(mini?)-computer revisited in 2011? Advice?

#1 Unread post by jpringle »

Dear Colleagues --

For the last 6 years, my ocean modeling needs have been well met by a 6 node cluster; each node a dual Opteron-250 with a single core for a total of 12 cores. This system has enough power to meet my modest process-oriented modeling needs; typical runs for me have a grid size of 380 by 90 by 25 and run for 200,000 timesteps. For the last 5 years, it was rock solid. However, it is starting to get old, and I am spending more time replacing power supplies and the like, and less time doing model runs.

I am thinking about how to replace it with a system of comparable power. I am attracted to a single box with dual processors and many cores so that I don't have to deal with the complexity and cost of a cluster (interconnects, etc). I also think that being able to use open-mp will allow us to tile the problem in a way that is more cache-friendly than using MPI on a cluster with ROMS 3.

In the past, I have been guided by Sasha Shchepetikin poorman's super-computer documents, and more recently his posts in viewtopic.php?f=17&t=2001&p=7771 . I am considering two systems: dual 6-core 2.6Ghz Xeons (X5650) or dual 12-core 2.2Ghz 12 core Opterons (6174).
  • Is there a different sort of system I should be considering?
  • Does anyone have experience with these systems? Do dual core systems of this sort have enough memory bandwidth to keep the cores fed?
  • I am particularly concerned that the Opterons may have insufficient memory bandwidth to keep the cores supplied with data. Can this be alleviated by choosing appropriate tile sizes and using open-mp?
  • Is the Intel compiler still the compiler of choice for these systems? For both the Opterons and Xeons?
  • For my size problem, is the "big-box" single computer preferable to a small cluster?
  • Are there any other gotchas to consider in specing such a system (For example, Sasha has stressed the importance of making sure the memory modules match the memory channels for each system).
    • Are there some motherboards to be sought/avoided?
    • Is the choice of memory (and its timings, etc) important?
    • What is the best linux kernel/distributions for running ROMS? (I don't care about deskstop stuff; I am not sure I would even run an Xserver on it.) I am most concerned about getting a kernel optimized for throughput and not interactivity.
Cheers,
and thanks,
Jamie Pringle
University of New Hampshire

Post Reply