Hey there,
It’s Robin from CFD Engine & here’s a question from one of last month’s Office Hours…
“I’ve been doing all my CFD on a local cloud provider, am I missing out? Does it have to be AWS?”
Short Answer
I talk about AWS a fair bit, so I can see how you might get the impression that it’s AWS or bust, but it’s not.
If you’ve got access to some well priced compute, plus storage (and they’ll let you install OpenFOAM on it) then go for it. Whether it’s in your neighbour’s garage or in an ultra-efficient data centre, if it works for you, for now, then use it.
See you next week 👋
Longer Answer
I want you to do the best CFD you can, within your resources. If your compute resources get a bit tight, then I want to have demonstrated that the cloud (not just AWS) is one possible solution.
In that spirit, I thought I’d share a few helpful bits for when you’re next thinking about running some (or all) of your OpenFOAM jobs in the cloud.
Single Instances
If you just want to offload some simulations to a remote workstation or two, then you’re in luck 🙌
Any cloud computing provider will rent you this kind of hardware by the hour.
It’s commodity computing – decide how many cores you want, how much RAM you need & pick the cheapest provider. If there’s a tie, choose the one that’s the most generous with their storage.
Cash for “Startups”
Speaking of “most generous” – if your company looks anything like a startup, then you might be eligible for free credits, training & support.
AWS have their Activate programme, there’s also Google Cloud for Startups & Microsoft for Startups.
Apparently Oracle Cloud also have a startup scheme, but I couldn’t find a working link to it 🤷♂️
If you apply (& qualify) then you can receive a decent chunk of store credit from these schemes, certainly more than enough to cover a few CFD simulations.
Multi-Cloud
Whilst I’m mentioning multiple clouds, by all means try a few of them, but at some point you’ll want to just pick one & go with it. The operational complexity of trying to maintain your workflow on multiple clouds rarely justifies any savings you might make.
If you really want to run on several clouds then check out what YellowDog & Rescale have to offer.
Clusters
When it comes to building clusters of instances & doing something more HPC-like, not all clouds are created equal.
Rescale pride themselves on their HPC credentials, as do several others, but it’s a bit more do-it-yourself in the public cloud.
AWS has a tool called ParallelCluster to automate building clusters on their hardware & Google Cloud have their Cloud HPC toolkit which does the same thing.
It’s a bit more difficult on Microsoft Azure, but they have an HPC repository with info & examples of the different cluster architectures you can build over there.
Regional Computing
Before jumping on the public cloud, it might be worth checking whether your municipality, region or local university have HPC facilities that you can access. MeluXina in Luxembourg is a great example of this. You may have something similar in your area, it’s worth a search.
Eco
All of the cloud providers would like you to believe that their offering is the greenest, but who really knows? I’ve got no idea if any one provider is better than another, or if at our usage levels you can even discern a difference. But there are a growing number of cloud providers that are making their green credentials their primary selling point.
From building their data centres in Iceland, to using otherwise wasted heat to warm swimming pools & apartments, to filling an old Norwegian mine with computers & cooling it with fjord water, there are some really interesting offerings if you’re looking to lower the carbon intensity of your computing.
It doesn’t have to be AWS
I like AWS for running OpenFOAM & I think you’d like it too, but it’s not the only game in town.
I want you to do your best CFD – and when you’re doing so much that you run out of compute power, I want you to know that there are plenty of ways to solve that problem.
Whether your computer is under your desk, in your neighbour’s garage or down a disused mine, I’m interested to hear where you’re running your CFD right now – drop me a note.
Until next week, stay safe,
