Hey there,
It’s Robin from CFD Engine & this week we’re scripting all the things.
Hot on the heels of last week’s “scripting ParaView” email, this week I’m breaking down what’s in my run script.
Some of it will be familiar, some of it you’ll never need, but hopefully it’s all interesting 🤞
There are a few AWS/cloud-specific bits, but they can easily be adapted if your cluster is in the next room.
This isn’t an in-depth teardown, it’s just a sketch, with the focus on those bits that might not be in your run scripts. Let’s get it…
#!/usr/bin/env bash
A quick note, I wrap all these bits up into a single Allrun bash script, an overgrown cousin of the ones found in the OpenFOAM tutorials.
When I start a new job, I log on to the instance, execute the run script & leave it to get on with it. Could be fancier, but it works pretty well.
Resource Tagging
The script’s first task is to parse info about the current run & tag the AWS resources that I’m using (via the AWS CLI tools). I use these tags to report compute costs per project, per run, per customer or any variation thereof. I don’t run the reports often, but they can be really eye opening.
Downloading Geometry
I usually keep the input geometries separate to the case directories, to avoid repeated uploads. So, before I can run the case I need to transfer the geometry files to the instance. The script determines which OBJ files are required & copies them over from S3, using the AWS CLI tools.
OpenFOAM stuff
It wouldn’t be much of a run script if we didn’t run some CFD.
I use the OpenFOAM run functions (runApplication & runParallel) to execute each step of the process.
It’s personal preference, but I like the named logs, the fact that it won’t run if a log already exists & that it picks up on the number of processors requested in decomposePar (I don’t need to specify the number of processors in the run script).
Other than that, it’s pretty standard fare really.
Exports & Batch Post-Processing
I export all the post-processing assets (slices, surfaces, streamlines etc) using postProcess and use the Python scripts we discussed last week, plus pvbatch, to generate all of the post-processing images that I need.
Making Movies
I use ffmpeg to convert the images from ParaView into MP4 files directly on the command-line. Super-fast but with one of the longest, most complicated run commands in the script – thankfully I don’t need to type it very often.
Upload to Dropbox
I often share data & post-processing with clients via Dropbox & so I added this step to reduce the amount of time taken to download bits from AWS & then re-upload them to Dropbox.
Instead, I upload the post-pro directly to Dropbox from my EC2 instance, using the Dropbox HTTP API.
It’s not a fancy integration, but it works & it’s really nice to have results waiting in Dropbox when you get back to your laptop.
Averaging Metrics
Almost all of my projects revolve around a force metric (usually drag &/or downforce) but intake pressures and mass-flow rates are in the mix too. I monitor a lot of metrics & I use a simple shell script to loop through & average them over a given number of iterations.
I typically use the forces functionObject (as opposed to forceCoeffs) and the file format is interesting to say the least. This step means that I don’t have to wrestle the output files in a spreadsheet, which is very welcome.
Archiving
The last major step, I archive the run to long-term storage (AWS S3 in my case) using the AWS CLI tools.
I archive the dictionaries, mesh & final timestep in a zipped tar archive.
I store the logs & force histories separate from the main archive as they often come in handy for debugging/checking cases. This way I don’t need to download the whole case archive to get at them.
I don’t archive most of the post-processing assets as they’re easily re-created. I make an exception for the surfaces of the car/bike/bus etc. They’re relatively small (compared to a full case) and it can be useful to pull them back to base occasionally.
I should probably stop doing this & save myself some storage dollars 🤔
Email summary
Before the instance shuts down, I have the run script email me a summary of the key metrics for the simulation, both so that I know it’s finished & so that I can add them to the results spreadsheet without needing to download anything else.
I use the Mailgun API (using cURL from the command-line) but if I were starting again I’d probably use the AWS Simple Email Service (SES) via their CLI tools.
Shutdown
Finally, the golden rule of AWS “turn it off when you’re not using it” – may not be a popular step if you’re running on a shared local machine 😬
That’s all folks
This is an outline sketch of the steps in my typical Allrun script. If there’s a step you want to know more about, or if you have a question on any of it – drop me a note – my inbox is always open & your responses make this worthwhile.
What special bits do you have in your run scripts? I’d love to steal hear more about them. Conversely, is there something you’d like your run script to do, but you’re not sure how to do it? Let me know, I might be able to help.
Until next week, CFD safely,
