Your Supercomputer Arrives In The Cloud

For as long as there have been supercomputers, people like us have seen the announcements and said, “Boy! I’d love to get some time on that computer.” But now that most of us have computers and phones that greatly outpace a Cray 2, what are we doing with them? Of course, a supercomputer today is still bigger than your PC by a long shot, and if you actually have a use case for one, [Stephen Wolfram] shows you how you can easily scale up your processing by borrowing resources from the Wolfram Compute Services. It isn’t free, but you pay with Wolfram service credits, which are not terribly expensive, especially compared to buying a supercomputer.

[Stephen] says he has about 200 cores of local processing at his house, and he still sometimes has programs that run overnight. If your program already uses a Wolfram language and uses parallelism — something easy to do with that toolbox — you can simply submit a remote batch job.

What constitutes a supercomputer? You get to pick. You can just offload your local machine using a single-core 8GB virtual machine — still a supercomputer by 1980s standards.  Or you get machines with up to 1.5TB of RAM and 192 cores. Not enough for your mad science? No worries, you can map a computation across more than one machine, too.

As an example, [Stephen] shows a simple program that tiles pentagons:

When the number of pentagons gets large, a single line of code sends it off to the cloud:

RemoteBatchSubmit[PentagonTiling[500]]

The basic machine class did the work in six minutes and 30 seconds for a cost of 5.39 credits. He also shows a meatier problem running on a 192-core 384GB machine. That job took less than two hours and cost a little under 11,000 credits (credit cost from just over $4/1000 to $6/1000, depending on how many you buy, so this job cost about $55 to run). If two hours is too much, you can map the same job across many small machines, get the answer in a few minutes, and spend fewer credits in the process.

Supercomputers today are both very different from old supercomputers and yet still somewhat the same. If you really want that time on the Cray you always wanted, you might think about simulation.

2 thoughts on “Your Supercomputer Arrives In The Cloud

  1. (Have not looked at it yet…) Given Wolfram this is no toy, but I wonder if it’s still a play space compared to a big boy machine.
    It’s not hard (or expensive) to spin up a few hundred cores and terabytes of memory on demand in Amazon’s EC2, for example. Or many tens of thousands, if you want NVIDIA cores.

    1. Quick lookup on EC2: 192 CPU cores, 135,168 GPU cores (8xH200), 3 TB of total memory is $75/hr. That 31,000 TFLOPS makes it comfortably in the top 100 supercomputer class.

      $75/hr would be just the electricity cost for that in a lot of places.

Leave a Reply

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.