Hey there,
It’s Robin from CFD Engine & it’s time for another trio of tiny OpenFOAM tips. No tutorials, no deep-dives, just three OpenFOAM nuggets that might’ve escaped your attention & that you might want to try out.
This time around, we have a tiny tool for grabbing info about your surface mesh, an easy way to make snappyHexMesh less memory hungry & an alternative decomposition method that can be tailored to your hardware.
All three have been around for a while & are available in both ESI/OpenCFD & Foundation versions – let’s go…
distributedTriSurfaceMesh
I only revisited this one after multiple prompts from Wouter & Nikola at AirShaper & I’m glad I did.
If you run out of memory while using snappyHexMesh in parallel, then this change could give you some extra headroom. Let me explain…
One of the little quirks of snappyHexMesh is that it stores a complete copy of your input geometry, in memory, for each core you use. This can escalate pretty quickly, especially when your input geometry has a lot of faces &/or you mesh on lots of cores.
ditributedTriSurfaceMesh is an alternative snappyHexMesh surface type, which allows SHM to partition your input geometry, sharing it across the cores & bringing the memory back to something more manageable.
Search-&-replace triSurfaceMesh with distributedTriSurfaceMesh in your snappyHexMeshDict & you’re almost done.
You also need to use hierarchical decomposition, but we know that’s the faster option for snappyHexMesh anyway.
Just for fun, I dug out a portly old project whose input geometry was around 7.5M faces. On 48-cores this model needed 31GB of memory before it even started mesh refinement. Switching to distributedTriSurfaceMesh brought that down to just 2GB.
I’ll take that (& then use the “spare” memory to add some more cells) 🤦♂️
Yu can find the docs here, should you need them.
surfaceMeshInfo
Do you need some basic info about a surface mesh? Then point surfaceMeshInfo at the OBJ/STL file & it’ll come back with the number of points, number of faces and the surface area, quick as a flash.
It can also output the area of each individual face & even dump it all into a weird XML format, should you need those features.
Overall: 10/10, would recommend to anyone needing to know the number of faces in their input geometry, from the command line, right now 👍
multiLevel Decomposition
In the previous "hierarchical is faster than scotch in snappyHexMesh" email I mentioned (in passing) a decomposition option that lets you combine several different methods to try & match your hardware – it’s called multiLevel decomposition & here’s a bit more info.
Imagine we had a cluster with ten nodes, each with two, 16-core CPUs (320-cores in total), we could express this in our decomposeParDict as:
It doesn’t have to be scotch for each level either. The documentation shows hierarchical for nodes, scotch for CPUs & metis for cores. I’ve no idea if that’s “optimal” but it opens up A LOT of tuning options & probably warrants some testing.
I’ve heard good things about this for big cases across lots of nodes (thanks Ivan) but my cases are typically on a single-instance so it’s not really a fit for me.
How about you? Have you tried it? Did you see a performance boost? Or are you going to take it for a quick spin? Let me know, I’d love to hear your experience with this one 🤔
That’s all folks
So there we have it – an easy way to reduce the memory that snappyHexMesh needs in parallel, a CLI tool to grab some stats from your input geometry & an alternative decomposition method for big jobs on lots of nodes.
They’re all relatively old, so perhaps you already know them? Do you use them? Or are they new-to-you & you’re planning to try them out? Let me know.
As always, drop me a note if you have an OpenFOAM nugget that you’d like me to share & I’ll add it to the list.
Until next week, stay safe 💛
