diff --git a/manual/sphinx/user_docs/adios2.rst b/manual/sphinx/user_docs/adios2.rst index d8e0135c0d..a39fe4a881 100644 --- a/manual/sphinx/user_docs/adios2.rst +++ b/manual/sphinx/user_docs/adios2.rst @@ -34,6 +34,10 @@ or on the BOUT++ command line set ``output:type=adios``. The default prefix is "BOUT.dmp" so the ADIOS file will be called "BOUT.dmp.bp". To change this, set the ``output:prefix`` option. +As with NetCDF output, BOUT++ will by default refuse to overwrite an +existing dump file. Use ``append`` to add new output to an existing +ADIOS dataset, or ``replace`` to overwrite it. + Restart files ------------- @@ -43,3 +47,4 @@ so to read and write restarts from an ADIOS dataset, put in BOUT.inp:: [restart_files] type = adios +Restart files still overwrite existing files by default. diff --git a/manual/sphinx/user_docs/bout_options.rst b/manual/sphinx/user_docs/bout_options.rst index 3d25ac1a14..02a6ed29f8 100644 --- a/manual/sphinx/user_docs/bout_options.rst +++ b/manual/sphinx/user_docs/bout_options.rst @@ -568,7 +568,9 @@ in table :numref:`tab-outputopts`. | Option | Description | Default value | +======================+=========================================+================+ | ``append`` | Append to existing file if true, | ``false`` | - | | otherwise overwrite | | + | | otherwise create a new file | | + +----------------------+-----------------------------------------+----------------+ + | ``replace`` | Overwrite an existing dump file if true | ``false`` | +----------------------+-----------------------------------------+----------------+ | ``enabled`` | Writing is enabled | ``true`` | +----------------------+-----------------------------------------+----------------+ @@ -587,6 +589,9 @@ in table :numref:`tab-outputopts`. - ``enabled`` is useful mainly for doing performance or scaling tests, where you want to exclude I/O from the timings. +- By default BOUT++ will not overwrite existing dump files. Set ``append=true`` + to add new time points to an existing file, or ``replace=true`` to overwrite it. + Restart files still overwrite existing files by default. - If you find that IO is taking more and more time as your simulation goes on, try setting ``flush_frequency`` to a larger value such as ``10``. This can workaround an issue with NetCDF where subsequent writes take longer and @@ -882,6 +887,10 @@ NetCDF file:: bout::OptionsIO::create("settings.nc")->write(Options::root()); +This form creates or overwrites ``settings.nc`` if it already exists. +To prevent overwriting, pass ``replace=false`` explicitly when creating +the file. + and to read it in again:: Options data = bout::OptionsIO::create("settings.nc")->read(); @@ -954,6 +963,11 @@ automatically set the ``"time_dimension"`` attribute:: // Append data to file bout::OptionsIO::create({{"file", "time.nc"}, {"append", true}})->write(data); +If you want to overwrite an existing file instead of appending, use +``replace=true``. If neither ``append`` nor ``replace`` is set, then +writing to an existing dump/output file will fail with an error rather +than silently overwriting data. + .. note:: By default, `bout::OptionsIO::write` will only write variables with a ``"time_dimension"`` of ``"t"``. You can write variables with a different time dimension by passing it as diff --git a/manual/sphinx/user_docs/running_bout.rst b/manual/sphinx/user_docs/running_bout.rst index 2b2b4c8ef8..268af7290c 100644 --- a/manual/sphinx/user_docs/running_bout.rst +++ b/manual/sphinx/user_docs/running_bout.rst @@ -96,12 +96,20 @@ Restart files allow the run to be restarted from where they left off:: $ mpirun -np 2 ./conduction restart -This will delete the output data ``BOUT.dmp.*.nc`` files, and start -again. If you want to keep the output from the first run, add “append”:: +By default, this will keep the existing output data ``BOUT.dmp.*.nc`` +files and then stop with an error rather than overwrite them. To +continue from the restart files and add new time points to the existing +output, add ``append``:: $ mpirun -np 2 ./conduction restart append -which will then append any new outputs to the end of the old data files. +To continue from the restart files and overwrite the existing output, +add ``replace``:: + + $ mpirun -np 2 ./conduction restart replace + +which will replace the old dump files with new output from the restarted +run. Restart files are still overwritten by default. For more information on restarting, see :ref:`sec-restarting`. To see some of the other command-line options try "-h":: @@ -538,14 +546,24 @@ of the command, for example:: $ mpirun -np 2 ./conduction restart -Equivalently, put “restart=true” near the top of the BOUT.inp input -file. Note that this will overwrite the existing data in the -``BOUT.dmp.\*.nc`` files. If you want to append to them instead then add -the keyword append to the command, for example:: +Equivalently, put ``restart=true`` near the top of the ``BOUT.inp`` input +file. By default, BOUT++ will refuse to overwrite existing +``BOUT.dmp.\*.nc`` files and will stop with an error instead. If you want +to append to them then add the keyword ``append`` to the command, for example:: $ mpirun -np 2 ./conduction restart append -or also put ``append=true`` near the top of the BOUT.inp input file. +or put ``append=true`` near the top of the ``BOUT.inp`` input file. + +If you want to overwrite the existing dump files, add ``replace`` to the +command:: + + $ mpirun -np 2 ./conduction restart replace + +or put ``replace=true`` near the top of the ``BOUT.inp`` input file. + +Restart files are not affected by this change: they are still overwritten +by default so that restarting continues to update the saved restart state. When restarting simulations BOUT++ will by default output the initial state, unless appending to existing data files when it will not output diff --git a/src/sys/options/options_adios.cxx b/src/sys/options/options_adios.cxx index 641a6994a8..24f9e93187 100644 --- a/src/sys/options/options_adios.cxx +++ b/src/sys/options/options_adios.cxx @@ -24,9 +24,11 @@ #include #include #include -#include +#include #include +namespace fs = std::filesystem; + namespace { std::size_t getLocalNDims(adios2::IO& io, const std::string& name, const adios2::Dims& shape) { @@ -233,6 +235,9 @@ OptionsADIOS::OptionsADIOS(Options& options) : OptionsIO(options) { ? adios2::Mode::Append : adios2::Mode::Write; + replace_existing_file = + options["replace"].doc("Replace existing file?").withDefault(false); + singleWriteFile = options["singleWriteFile"].withDefault(false); } @@ -404,6 +409,17 @@ namespace bout { void OptionsADIOS::write(const Options& options, const std::string& time_dim) { const Timer timer("io"); + if (file_mode == adios2::Mode::Write and fs::exists(filename) + and !replace_existing_file) { + throw BoutException( + "Output file '{:s}' already exists. Aborting rather than overwriting data.\n" + "Set 'append=true' to append, or 'replace=true' to overwrite.", + filename); + } + // Write gets called multiple times. + // Only prevent overwrite the first time. + replace_existing_file = true; + // ADIOSStream is just a BOUT++ object, it does not create anything inside ADIOS ADIOSStream& stream = ADIOSStream::ADIOSGetStream(filename, file_mode); diff --git a/src/sys/options/options_adios.hxx b/src/sys/options/options_adios.hxx index 3a67c17e7f..4e454b9ac1 100644 --- a/src/sys/options/options_adios.hxx +++ b/src/sys/options/options_adios.hxx @@ -67,6 +67,7 @@ private: std::string filename; /// How to open the file for writing adios2::Mode file_mode{adios2::Mode::Write}; + bool replace_existing_file; ///< Replace existing file if it exists? bool singleWriteFile = false; }; diff --git a/src/sys/options/options_io.cxx b/src/sys/options/options_io.cxx index 256ec56b28..41d61be5de 100644 --- a/src/sys/options/options_io.cxx +++ b/src/sys/options/options_io.cxx @@ -2,11 +2,15 @@ #include "bout/bout.hxx" #include "bout/globals.hxx" #include "bout/mesh.hxx" +#include "bout/options.hxx" #include "bout/version.hxx" #include "options_adios.hxx" #include "options_netcdf.hxx" +#include +#include + namespace bout { std::unique_ptr OptionsIO::create(const std::string& file) { return OptionsIOFactory::getInstance().createFile(file); @@ -25,6 +29,7 @@ OptionsIOFactory::ReturnType OptionsIOFactory::createRestart(Options* optionsptr Options::root()["datadir"].withDefault("data")); options["prefix"].overrideDefault("BOUT.restart"); options["append"].overrideDefault(false); + options["replace"].overrideDefault(true); // Restart files are overwritten each output options["singleWriteFile"].overrideDefault(true); return create(getType(&options), options); } @@ -39,11 +44,15 @@ OptionsIOFactory::ReturnType OptionsIOFactory::createOutput(Options* optionsptr) options["append"].overrideDefault(Options::root()["append"] .doc("Add output data to existing (dump) files?") .withDefault(false)); + options["replace"].overrideDefault( + Options::root()["replace"] + .doc("Replace output (dump) files if they exist and not appending?") + .withDefault(false)); return create(getType(&options), options); } OptionsIOFactory::ReturnType OptionsIOFactory::createFile(const std::string& file) const { - Options options{{"file", file}}; + Options options{{"file", file}, {"replace", true}}; return create(getDefaultType(), options); } diff --git a/src/sys/options/options_netcdf.cxx b/src/sys/options/options_netcdf.cxx index 9170f7c284..d915b02b00 100644 --- a/src/sys/options/options_netcdf.cxx +++ b/src/sys/options/options_netcdf.cxx @@ -4,17 +4,30 @@ #include "options_netcdf.hxx" +#include "bout/array.hxx" +#include "bout/assert.hxx" #include "bout/bout.hxx" #include "bout/bout_types.hxx" +#include "bout/boutexception.hxx" #include "bout/mesh.hxx" +#include "bout/options_io.hxx" #include "bout/sys/timer.hxx" +#include "bout/sys/variant.hxx" #include "bout/traits.hxx" +#include "bout/unused.hxx" +#include "bout/utils.hxx" #include #include #include +#include +#include +#include +#include #include +#include +#include #include using namespace netCDF; @@ -206,7 +219,7 @@ void readGroup(const std::string& filename, const NcGroup& group, Options& resul namespace bout { Options OptionsNetCDF::read(bool lazy) { - Timer timer("io"); + const Timer timer("io"); // Open file auto read_file = std::make_shared(filename, NcFile::read); @@ -399,7 +412,7 @@ struct NcPutVarVisitor { template <> void NcPutVarVisitor::operator()(const bool& value) { - int int_val = value ? 1 : 0; + const int int_val = value ? 1 : 0; var.putVar(&int_val); } @@ -494,7 +507,7 @@ struct NcPutAttVisitor { template <> void NcPutAttVisitor::operator()(const bool& value) { - int ival = value ? 1 : 0; + const int ival = value ? 1 : 0; var.putAtt(name, ncInt, ival); } template <> @@ -751,9 +764,17 @@ OptionsNetCDF::OptionsNetCDF(Options& options) : OptionsIO(options) { options["prefix"].as(), BoutComm::rank()); } - file_mode = (options["append"].doc("Append to existing file?").withDefault(false)) - ? FileMode::append - : FileMode::replace; + const bool appending = + options["append"].doc("Append to existing file?").withDefault(false); + const bool replacing = + options["replace"].doc("Replace existing file?").withDefault(false); + if (appending) { + file_mode = FileMode::append; + } else if (replacing) { + file_mode = FileMode::replace; + } else { + file_mode = FileMode::newFile; + } } void OptionsNetCDF::verifyTimesteps() const { @@ -778,17 +799,34 @@ void OptionsNetCDF::verifyTimesteps() const { /// Write options to file void OptionsNetCDF::write(const Options& options, const std::string& time_dim) { - Timer timer("io"); + const Timer timer("io"); // Check the file mode to use auto ncmode = NcFile::replace; - if (file_mode == FileMode::append) { + switch (file_mode) { + case FileMode::append: { // NetCDF doesn't have a "read-write, create if exists" mode, so // we need to check ourselves if the file already exists; if it // doesn't, tell NetCDF to create it - std::ifstream file(filename); + const std::ifstream file(filename); ncmode = file.good() ? NcFile::FileMode::write : NcFile::FileMode::newFile; + break; } + case FileMode::replace: { + // Replacing the file if it exists + break; + } + case FileMode::newFile: + // Creating a new file. Don't overwrite existing data. + const std::ifstream file(filename); + if (file.good()) { + throw BoutException( + "Output file '{:s}' already exists. Aborting rather than overwriting data.\n" + "Set 'append=true' to append, or 'replace=true' to overwrite.", + filename); + } + break; + }; if (not data_file) { data_file = std::make_unique(filename, ncmode); diff --git a/src/sys/options/options_netcdf.hxx b/src/sys/options/options_netcdf.hxx index 3e2a69b6d7..bf2c69bcab 100644 --- a/src/sys/options/options_netcdf.hxx +++ b/src/sys/options/options_netcdf.hxx @@ -63,6 +63,7 @@ public: private: enum class FileMode { + newFile, ///< Create a new file. Do not overwrite or append replace, ///< Overwrite file when writing append ///< Append to file when writing }; diff --git a/tests/MMS/diffusion/runtest b/tests/MMS/diffusion/runtest index ef6b335f70..b76b631001 100755 --- a/tests/MMS/diffusion/runtest +++ b/tests/MMS/diffusion/runtest @@ -2,18 +2,11 @@ # Python script to run and analyse MMS test -from __future__ import division -from __future__ import print_function +import sys -try: - from builtins import str -except: - pass - -from boututils.run_wrapper import shell, build_and_log, launch_safe from boutdata.collect import collect - -from numpy import sqrt, max, abs, mean, array, log +from boututils.run_wrapper import build_and_log, launch_safe, shell +from numpy import abs, array, log, max, mean, sqrt build_and_log("MMS diffusion test") @@ -29,7 +22,16 @@ error_2 = [] # The L2 error (RMS) error_inf = [] # The maximum error for nx in nxlist: - args = "mesh:nx=" + str(nx) + " nout=" + str(nout) + " timestep=" + str(timestep) + # Set the mesh size, ouput steps and timestep. Replace previous outputs if present. + args = ( + "mesh:nx=" + + str(nx) + + " nout=" + + str(nout) + + " timestep=" + + str(timestep) + + " replace" + ) print("Running with " + args) @@ -42,9 +44,8 @@ for nx in nxlist: s, out = launch_safe(cmd, nproc=nproc, pipe=True) # Save output to log file - f = open("run.log." + str(nx), "w") - f.write(out) - f.close() + with open("run.log." + str(nx), "w") as f: + f.write(out) # Collect data E_N = collect("E_N", tind=[nout, nout], path="data", info=False) @@ -58,13 +59,13 @@ for nx in nxlist: error_2.append(l2) error_inf.append(linf) - print("Error norm: l-2 %f l-inf %f" % (l2, linf)) + print(f"Error norm: l-2 {l2} l-inf {linf}") # Calculate grid spacing dx = 1.0 / (array(nxlist) - 2.0) order = log(error_2[-1] / error_2[-2]) / log(dx[-1] / dx[-2]) -print("Convergence order = %f" % (order)) +print(f"Convergence order = {order}") # Attempt to plot errors try: @@ -72,9 +73,7 @@ try: plt.plot(dx, error_2, "-o", label=r"$l^2$") plt.plot(dx, error_inf, "-x", label=r"$l^\infty$") - plt.plot( - dx, error_2[-1] * (dx / dx[-1]) ** order, "--", label="Order %.1f" % (order) - ) + plt.plot(dx, error_2[-1] * (dx / dx[-1]) ** order, "--", label=f"Order {order}") plt.legend(loc="upper left") plt.grid() @@ -89,16 +88,16 @@ try: # plt.show() plt.close() -except: +except Exception as e: # noqa: BLE001 # Plotting could fail for any number of reasons, and the actual # error raised may depend on, among other things, the current # matplotlib backend, so catch everything - pass + print(f"Plotting: Caught exception {e}") if order > 1.8 and order < 2.2: # test for success print(" => Test passed") - exit(0) + sys.exit(0) else: print(" => Test failed") - exit(1) + sys.exit(1) diff --git a/tests/MMS/spatial/finite-volume/runtest b/tests/MMS/spatial/finite-volume/runtest index bcd4672545..0a9e15cd6e 100755 --- a/tests/MMS/spatial/finite-volume/runtest +++ b/tests/MMS/spatial/finite-volume/runtest @@ -70,8 +70,8 @@ def assert_convergence(error, dx, name, expected) -> bool: def run_fv_operators(nz: int) -> dict[str, float]: - # Command to run - cmd = f"./fv_mms MZ={nz} mesh:ny={nz}" + # Command to run. Replace previous outputs if present + cmd = f"./fv_mms MZ={nz} mesh:ny={nz} replace" print(f"Running command: {cmd}", end="") # Launch using MPI diff --git a/tests/MMS/wave-1d-y/runtest b/tests/MMS/wave-1d-y/runtest index 332d6864b0..64fc335389 100755 --- a/tests/MMS/wave-1d-y/runtest +++ b/tests/MMS/wave-1d-y/runtest @@ -2,23 +2,14 @@ # Python script to run and analyse MMS test -from __future__ import division -from __future__ import print_function - -try: - from builtins import zip - from builtins import str -except: - pass - -from boututils.run_wrapper import shell, build_and_log, launch_safe -from boutdata.collect import collect import pickle - +import sys from sys import stdout -from numpy import sqrt, max, abs, mean, array, log, concatenate, pi +from boutdata.collect import collect +from boututils.run_wrapper import build_and_log, launch_safe, shell +from numpy import abs, array, log, max, mean, pi, sqrt build_and_log("Making MMS wave test") @@ -51,6 +42,7 @@ for ny in nylist: + str(nout) + " timestep=" + str(timestep) + + " replace" ) print("Running with " + args) @@ -80,7 +72,7 @@ for ny in nylist: error_2[var].append(l2) error_inf[var].append(linf) - print("Error norm %s: l-2 %f l-inf %f" % (var, l2, linf)) + print(f"Error norm {var}: l-2 {l2} l-inf {linf}") # Save data with open("wave.pkl", "wb") as output: @@ -95,7 +87,7 @@ dy = 1.0 / array(nylist) success = True for var in varlist: order = log(error_2[var][-1] / error_2[var][-2]) / log(dy[-1] / dy[-2]) - stdout.write("%s Convergence order = %f" % (var, order)) + stdout.write(f"{var} Convergence order = {order}") if 1.8 < order < 2.2: # Should be second order accurate print("............ PASS") @@ -108,7 +100,7 @@ try: import matplotlib.pyplot as plt for var, mark, label in zip(varlist, markers, labels): - plt.plot(dy, error_2[var], "-" + mark, label="%s order=%.2f" % (label, order)) + plt.plot(dy, error_2[var], "-" + mark, label=f"{label} order={order}") plt.plot(dy, error_inf[var], "--" + mark) plt.legend(loc="upper left") @@ -124,15 +116,15 @@ try: # plt.show() plt.close() -except: +except Exception as e: # noqa: BLE001 # Plotting could fail for any number of reasons, and the actual # error raised may depend on, among other things, the current # matplotlib backend, so catch everything - pass + print(f"Plotting: Caught exception {e}") if success: print(" => Test passed") - exit(0) + sys.exit(0) else: print(" => Test failed") - exit(1) + sys.exit(1) diff --git a/tests/MMS/wave-1d/runtest b/tests/MMS/wave-1d/runtest index 1adb7c7d6d..b775606b98 100755 --- a/tests/MMS/wave-1d/runtest +++ b/tests/MMS/wave-1d/runtest @@ -2,18 +2,11 @@ # Python script to run and analyse MMS test -from __future__ import division -from __future__ import print_function +import sys -try: - from builtins import str -except: - pass - -from boututils.run_wrapper import shell, build_and_log, launch_safe from boutdata.collect import collect - -from numpy import sqrt, max, abs, mean, array, log, concatenate +from boututils.run_wrapper import build_and_log, launch_safe, shell +from numpy import abs, array, concatenate, log, max, mean, sqrt build_and_log("Making MMS wave test") @@ -29,7 +22,16 @@ error_2 = [] # The L2 error (RMS) error_inf = [] # The maximum error for nx in nxlist: - args = "mesh:nx=" + str(nx) + " nout=" + str(nout) + " timestep=" + str(timestep) + # Set mesh size and outputs. Replace previous outputs if present. + args = ( + "mesh:nx=" + + str(nx) + + " nout=" + + str(nout) + + " timestep=" + + str(timestep) + + " replace" + ) print("Running with " + args) @@ -42,9 +44,8 @@ for nx in nxlist: s, out = launch_safe(cmd, nproc=nproc, pipe=True) # Save output to log file - f = open("run.log." + str(nx), "w") - f.write(out) - f.close() + with open("run.log." + str(nx), "w") as f: + f.write(out) # Collect data E_f = collect("E_f", tind=[nout, nout], path="data", info=False) @@ -62,13 +63,13 @@ for nx in nxlist: error_2.append(l2) error_inf.append(linf) - print("Error norm: l-2 %f l-inf %f" % (l2, linf)) + print(f"Error norm: l-2 {l2} l-inf {linf}") # Calculate grid spacing dx = 1.0 / (array(nxlist) - 2.0) order = log(error_2[-1] / error_2[-2]) / log(dx[-1] / dx[-2]) -print("Convergence order = %f" % (order)) +print(f"Convergence order = {order}") # plot errors try: @@ -76,9 +77,7 @@ try: plt.plot(dx, error_2, "-o", label=r"$l^2$") plt.plot(dx, error_inf, "-x", label=r"$l^\infty$") - plt.plot( - dx, error_2[-1] * (dx / dx[-1]) ** order, "--", label="Order %.1f" % (order) - ) + plt.plot(dx, error_2[-1] * (dx / dx[-1]) ** order, "--", label=f"Order {order:.1f}") plt.legend(loc="upper left") plt.grid() @@ -93,16 +92,16 @@ try: # plt.show() plt.close() -except: +except Exception as e: # noqa: BLE001 # Plotting could fail for any number of reasons, and the actual # error raised may depend on, among other things, the current # matplotlib backend, so catch everything - pass + print(f"Plotting: Caught exception {e}") if 2.2 > order > 1.8: # test for success print(" => Test passed") - exit(0) + sys.exit(0) else: print(" => Test failed") - exit(1) + sys.exit(1) diff --git a/tests/conftest.py b/tests/conftest.py index f04f85e562..43a4372e3e 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -1,10 +1,19 @@ -import pytest -import shutil import os +import shutil import subprocess import time from pathlib import Path +import pytest + + +def remove_bout_dmp_files(path: Path) -> None: + for entry in path.rglob("BOUT.dmp.*"): + if entry.is_dir(): + shutil.rmtree(entry) + elif entry.exists() or entry.is_symlink(): + entry.unlink() + def pytest_configure(config): config.addinivalue_line( @@ -34,21 +43,37 @@ def copy_and_cwd_to_unique_tmp_dir(request, tmp_path_factory, monkeypatch): run_dir = tmp_path_factory.mktemp(test_file_dir.name) # Copy the original test directory into it - shutil.copytree(test_file_dir, run_dir, dirs_exist_ok=True) + shutil.copytree( + test_file_dir, + run_dir, + dirs_exist_ok=True, + ignore=shutil.ignore_patterns("BOUT.dmp.*"), + ) # Change working directory to the copy monkeypatch.chdir(run_dir) +@pytest.fixture(scope="function", autouse=True) +def remove_bout_dmp_before_test(copy_and_cwd_to_unique_tmp_dir): + remove_bout_dmp_files(Path(".")) + + @pytest.fixture def assert_success_in_shell(test_dir): def inner_function(command: str): + remove_bout_dmp_files(Path(".")) # MPI oversubscribe for communications test os.environ["OMPI_MCA_rmaps_base_oversubscribe"] = "1" # Allows 18 procs start = time.time() result = subprocess.run( - command, shell=True, capture_output=True, text=True, timeout=600 + command, + shell=True, + capture_output=True, + text=True, + timeout=600, + check=False, ) elapsed = time.time() - start assert result.returncode == 0, ( diff --git a/tests/integrated/test-boutpp/conftest.py b/tests/integrated/test-boutpp/conftest.py index 09631a1fe6..940f3cf5ff 100644 --- a/tests/integrated/test-boutpp/conftest.py +++ b/tests/integrated/test-boutpp/conftest.py @@ -1,7 +1,18 @@ -import pytest import os -import sys +import shutil import subprocess +import sys +from pathlib import Path + +import pytest + + +def remove_bout_dmp_files(path: Path) -> None: + for entry in path.rglob("BOUT.dmp.*"): + if entry.is_dir(): + shutil.rmtree(entry) + elif entry.exists() or entry.is_symlink(): + entry.unlink() @pytest.fixture(autouse=True, scope="function") @@ -13,6 +24,11 @@ def unique_xdist_group(request): request.node.add_marker(pytest.mark.xdist_group(name=group_name)) +@pytest.fixture(autouse=True, scope="function") +def remove_boutpp_outputs(): + remove_bout_dmp_files(Path(".")) + + @pytest.fixture(scope="function") def run_isolated(request): """ @@ -52,7 +68,14 @@ def run_isolated(request): str(request.config.inifile or root_dir), # Point to actual config if it exists ] - result = subprocess.run(cmd, env=env, cwd=root_dir, capture_output=True, text=True) + result = subprocess.run( + cmd, + env=env, + cwd=root_dir, + capture_output=True, + text=True, + check=False, + ) if result.returncode != 0: pytest.fail( diff --git a/tests/integrated/test-communications/test_communications.py b/tests/integrated/test-communications/test_communications.py index 51beb1f537..9e4ac39dd0 100755 --- a/tests/integrated/test-communications/test_communications.py +++ b/tests/integrated/test-communications/test_communications.py @@ -1,9 +1,10 @@ #!/usr/bin/env python3 import os + import numpy as np import numpy.testing as npt from boututils.datafile import DataFile -from boututils.run_wrapper import shell, launch_safe +from boututils.run_wrapper import launch_safe, shell # Note, the expected values for the communicated cells, which are used to test the # results, are shown in corner_communication_diagram.ods @@ -37,11 +38,11 @@ def test_communications(): command = "./test-communications NXPE=" + str(nxpe) # remove old outputs - shell(["rm data/BOUT.dmp.*"]) + shell(["rm -rf data/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run.log." + str(nxpe), "w") as f: f.write(out) @@ -179,11 +180,11 @@ def test_communications(): command = "./test-communications NXPE=" + str(nxpe) # remove old outputs - shell(["rm data/BOUT.dmp.*"]) + shell(["rm -rf data/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run.log." + str(nxpe), "w") as f: f.write(out) @@ -279,11 +280,11 @@ def test_communications(): command = "./test-communications NXPE=" + str(nxpe) # remove old outputs - shell(["rm data/BOUT.dmp.*"]) + shell(["rm -rf data/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run.log." + str(nxpe), "w") as f: f.write(out) @@ -342,11 +343,11 @@ def test_communications(): command = "./test-communications -d data_limiter NXPE=" + str(nxpe) # remove old outputs - shell(["rm data_limiter/BOUT.dmp.*"]) + shell(["rm -rf data_limiter/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run_limiter.log." + str(nxpe), "w") as f: f.write(out) @@ -378,11 +379,11 @@ def test_communications(): command = "./test-communications -d data_limiter NXPE=" + str(nxpe) # remove old outputs - shell(["rm data_limiter/BOUT.dmp.*"]) + shell(["rm -rf data_limiter/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run_limiter.log." + str(nxpe), "w") as f: f.write(out) @@ -407,11 +408,11 @@ def test_communications(): command = "./test-communications -d data_limiter NXPE=" + str(nxpe) # remove old outputs - shell(["rm data_limiter/BOUT.dmp.*"]) + shell(["rm -rf data_limiter/BOUT.dmp.*"]) print("Running Communications Test, nproc=" + str(nxpe * nype)) - s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) + _s, out = launch_safe(command, nproc=nxpe * nype, pipe=True) with open("run_limiter.log." + str(nxpe), "w") as f: f.write(out) diff --git a/tests/integrated/test-cvode-save-jacobian/test_cvode_save_jacobian.py b/tests/integrated/test-cvode-save-jacobian/test_cvode_save_jacobian.py index 0e1bed10d5..138dfe4295 100644 --- a/tests/integrated/test-cvode-save-jacobian/test_cvode_save_jacobian.py +++ b/tests/integrated/test-cvode-save-jacobian/test_cvode_save_jacobian.py @@ -1,3 +1,4 @@ +import subprocess from pathlib import Path import numpy as np @@ -7,7 +8,12 @@ TEST_DIR = Path(".") +def remove_bout_dmp_files(): + subprocess.run("rm -rf data/BOUT.dmp.*", shell=True, check=False) + + def test_rhs_jacobian_output_trigger(assert_success_in_shell): + remove_bout_dmp_files() assert_success_in_shell("./test_cvode_save_jacobian") data_dir = TEST_DIR / "data" @@ -44,6 +50,7 @@ def test_rhs_jacobian_output_trigger(assert_success_in_shell): def test_system_jacobian_linear_setup_trigger(assert_success_in_shell): + remove_bout_dmp_files() assert_success_in_shell( "./test_cvode_save_jacobian " "solver:cvode_precon_method=petsc " diff --git a/tests/integrated/test-cyclic/test_cyclic.py b/tests/integrated/test-cyclic/test_cyclic.py index 19f78a48d9..507de3931c 100755 --- a/tests/integrated/test-cyclic/test_cyclic.py +++ b/tests/integrated/test-cyclic/test_cyclic.py @@ -4,9 +4,9 @@ # Run the test, check it completed successfully # -from boututils.run_wrapper import shell, launch from sys import stdout +from boututils.run_wrapper import launch, shell flags = ["", "nsys=2", "nsys=5 periodic", "nsys=7 n=10"] @@ -17,12 +17,12 @@ def test_cyclic(): for nproc in [1, 2, 4]: cmd = "./test_cyclic" - print(" %d processors...." % (nproc)) + print(f" {nproc} processors....") r = 0 for f in flags: stdout.write("\tflags '" + f + "' ... ") - shell(["rm data/BOUT.dmp.* 2> err.log"]) + shell(["rm -rf data/BOUT.dmp.* 2> err.log"]) # Run the case status, out = launch(cmd + " " + f, nproc=nproc, mthread=1, pipe=True) diff --git a/tests/integrated/test-datafilefacade/test_datafile_facade.py b/tests/integrated/test-datafilefacade/test_datafile_facade.py index 2f1e3a5187..0f561fa758 100755 --- a/tests/integrated/test-datafilefacade/test_datafile_facade.py +++ b/tests/integrated/test-datafilefacade/test_datafile_facade.py @@ -3,11 +3,10 @@ # Test the DataFileFacade interface by writing to dump files using the SAVE_ONCE macro # -import pytest import numpy +import pytest from boutdata.collect import collect -from boututils.run_wrapper import shell, launch_safe - +from boututils.run_wrapper import launch_safe, shell testvars = { "f2d": 0.1, @@ -36,12 +35,12 @@ def test_datafile_facade(nproc): # delete any existing output - shell(["rm -f data/BOUT.dmp.*.nc data/BOUT.restart.*.nc"]) + shell(["rm -rf data/BOUT.dmp.* data/BOUT.restart.*.nc"]) print(f" {nproc} processor(s)....") # run the test executable - s, out = launch_safe("./test-datafile-facade", nproc=nproc, pipe=True) + _s, out = launch_safe("./test-datafile-facade", nproc=nproc, pipe=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) @@ -52,7 +51,7 @@ def test_datafile_facade(nproc): if result.dtype.kind in ("S", "U"): assert str(result) == expected, ( - f"{name} is different: got '{str(result)}', expected '{expected}'" + f"{name} is different: got '{result!s}', expected '{expected}'" ) else: diff = numpy.max(numpy.abs(expected - result)) diff --git a/tests/integrated/test-delp2/test_delp2.py b/tests/integrated/test-delp2/test_delp2.py index 4f48974537..eff9073988 100755 --- a/tests/integrated/test-delp2/test_delp2.py +++ b/tests/integrated/test-delp2/test_delp2.py @@ -1,10 +1,11 @@ #!/usr/bin/env python3 -import pytest -import numpy as np from sys import stdout -from boututils.run_wrapper import shell, launch_safe + +import numpy as np +import pytest from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell tol = 1e-10 # Absolute tolerance @@ -28,7 +29,9 @@ def test_delp2(setting): print("Args: " + setting) cmd = exefile + " " + setting - s, out = launch_safe(cmd, nproc=1, pipe=True) + shell(["rm -rf data/BOUT.dmp.*"]) + + _s, out = launch_safe(cmd, nproc=1, pipe=True) file_suffix = ".".join([x.split("=")[-1] for x in setting.split()]) with open("run.log." + str(file_suffix) + ".1", "w") as f: f.write(out) @@ -36,10 +39,10 @@ def test_delp2(setting): n0 = collect("n", path="data", info=False) for nproc in [2, 4]: - shell(["rm data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) - stdout.write(" %d processor...." % (nproc)) - s, out = launch_safe(cmd, nproc=nproc, mthread=1, pipe=True) + stdout.write(f" {nproc} processor....") + _s, out = launch_safe(cmd, nproc=nproc, mthread=1, pipe=True) with open("run.log." + str(file_suffix) + "." + str(nproc), "w") as f: f.write(out) diff --git a/tests/integrated/test-drift-instability/test_drift_instability.py b/tests/integrated/test-drift-instability/test_drift_instability.py old mode 100644 new mode 100755 index 45b927fc59..7498e5581e --- a/tests/integrated/test-drift-instability/test_drift_instability.py +++ b/tests/integrated/test-drift-instability/test_drift_instability.py @@ -4,16 +4,15 @@ # Run the test, compare results against the benchmark # -import pytest -import numpy as np +from math import isnan -from boututils.run_wrapper import shell, launch_safe +import numpy as np +import pytest +from boutdata.collect import collect from boututils.calculus import deriv from boututils.datafile import DataFile from boututils.linear_regression import linear_regression - -from boutdata.collect import collect -from math import isnan +from boututils.run_wrapper import launch_safe, shell nthreads = 1 nproc = 2 # Number of processors to run on @@ -65,13 +64,13 @@ def run_zeff_case(zeff): timestep = 1e3 # Delete old output files - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print("Running drift instability test, zeff = ", zeff) # Run the case - s, out = launch_safe( - "./2fluid 2fluid:Zeff={} solver:output_step={}".format(zeff, timestep), + _s, out = launch_safe( + f"./2fluid 2fluid:Zeff={zeff} solver:output_step={timestep}", nproc=nproc, mthread=nthreads, pipe=True, @@ -93,7 +92,7 @@ def run_zeff_case(zeff): nz = dims[3] with DataFile("uedge.grd_std.cdl") as f: - grid = {v: f.read(v) for v in f.keys()} + grid = {v: f.read(v) for v in f} # Calculate geometric and physical quantities lZeta = 1e2 * zmax * 2 * np.pi * grid["R0"] # toroidal range [cm] diff --git a/tests/integrated/test-fci-mpi/test_fci_mpi.py b/tests/integrated/test-fci-mpi/test_fci_mpi.py index 05df0bf974..3fb0cb3094 100755 --- a/tests/integrated/test-fci-mpi/test_fci_mpi.py +++ b/tests/integrated/test-fci-mpi/test_fci_mpi.py @@ -2,11 +2,11 @@ # # Python script to run and analyse MPI test -from boututils.run_wrapper import launch_safe -from boutdata.collect import collect import itertools import numpy.testing as npt +from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell # Resolution in x and y NLIST = [1, 2, 4] @@ -16,13 +16,15 @@ def test_fci_mpi(): - COLLECT_KW = dict(info=False, xguards=False, yguards=False, path="data") + COLLECT_KW = {"info": False, "xguards": False, "yguards": False, "path": "data"} def run_case(nxpe: int, nype: int, mthread: int): cmd = f"./fci_mpi NXPE={nxpe} NYPE={nype} mesh:paralleltransform:xzinterpolation:type={implementation}" print(f"Running command: {cmd}") + shell(["rm -rf data/BOUT.dmp.*"]) + _, out = launch_safe(cmd, nproc=nxpe * nype, mthread=mthread, pipe=True) # Save output to log file diff --git a/tests/integrated/test-fieldgroupComm/test_fieldgroupComm.py b/tests/integrated/test-fieldgroupComm/test_fieldgroupComm.py index 7cf1347329..05835b00cb 100755 --- a/tests/integrated/test-fieldgroupComm/test_fieldgroupComm.py +++ b/tests/integrated/test-fieldgroupComm/test_fieldgroupComm.py @@ -5,8 +5,8 @@ # import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell from numpy import abs, seterr @@ -23,7 +23,7 @@ def test_fieldgroupcomm(nproc): cmd = f"./{exe_name}" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print(f"Running FieldGroup comm test on {nproc} processor(s)...") _, out = launch_safe(cmd, nproc=nproc, pipe=True) diff --git a/tests/integrated/test-griddata-yboundary-guards/test_griddata_yboundary_guards.py b/tests/integrated/test-griddata-yboundary-guards/test_griddata_yboundary_guards.py index 6f96acd8e2..f6a5be2995 100755 --- a/tests/integrated/test-griddata-yboundary-guards/test_griddata_yboundary_guards.py +++ b/tests/integrated/test-griddata-yboundary-guards/test_griddata_yboundary_guards.py @@ -1,11 +1,12 @@ #!/usr/bin/env python3 -import pytest -import numpy from pathlib import Path -from netCDF4 import Dataset -from boututils.run_wrapper import shell, launch_safe + +import numpy +import pytest from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell +from netCDF4 import Dataset @pytest.mark.parametrize("topology", ["doublenull", "singlenull"]) @@ -83,13 +84,13 @@ def test_griddata_yboundary_guards(topology, n_yguards): gridfile.createVariable("test", float, ("x", "y")) gridfile["test"][...] = testdata - shell([f"rm -f {datadir}/BOUT.dmp.*.nc run.log.*"]) + shell([f"rm -rf {datadir}/BOUT.dmp.* run.log.*"]) print( f"Running {topology} test with n_yguards={n_yguards} on {nproc} processors..." ) - s, out = launch_safe(f"./test_griddata -d {datadir}", nproc=nproc, pipe=True) + _s, out = launch_safe(f"./test_griddata -d {datadir}", nproc=nproc, pipe=True) with open(f"run.log.{topology}.{n_yguards}.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-gyro/test_gyro.py b/tests/integrated/test-gyro/test_gyro.py index d8a8978263..bd3ee44ac6 100755 --- a/tests/integrated/test-gyro/test_gyro.py +++ b/tests/integrated/test-gyro/test_gyro.py @@ -1,14 +1,15 @@ #!/usr/bin/env python3 from pathlib import Path +from sys import stdout + +import numpy as np # # Run the test, compare results against the benchmark # import pytest -import numpy as np -from sys import stdout -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell # Variables to compare variables = ["pade1", "pade2"] @@ -40,10 +41,10 @@ def test_gyro(benchmark_data): cmd = f"./test_gyro NXPE={nxpe}" - shell("rm -f data/BOUT.dmp.*.nc") + shell("rm -rf data/BOUT.dmp.*") - print(" %d processors (nxpe = %d)...." % (nproc, nxpe)) - s, out = launch_safe(cmd, nproc=nproc, pipe=True) + print(f" {nproc} processors (nxpe = {nxpe})....") + _s, out = launch_safe(cmd, nproc=nproc, pipe=True) with open("run.log." + str(nproc), "w") as f: f.write(out) diff --git a/tests/integrated/test-initial/test_initial.py b/tests/integrated/test-initial/test_initial.py index 3a803cc163..f71cd5f63b 100755 --- a/tests/integrated/test-initial/test_initial.py +++ b/tests/integrated/test-initial/test_initial.py @@ -2,17 +2,16 @@ # Test initial conditions -import pytest import configparser import itertools import platform import warnings -import numpy as np -from scipy.special import erf from pathlib import Path -from boututils.run_wrapper import shell, launch_safe +import numpy as np +import pytest from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell ######################################## # Implementations of BOUT++ functions @@ -135,7 +134,6 @@ def fmod(x, denominator=1.0): abs = np.abs asin = np.arcsin acos = np.arccos -ballooning = ballooning cos = np.cos cosh = np.cosh exp = np.exp @@ -149,7 +147,6 @@ def fmod(x, denominator=1.0): tan = np.tan TanhHat = tanhhat pi = np.pi -erf = erf @pytest.mark.parametrize("nproc", [1, 2, 3, 4]) @@ -159,7 +156,7 @@ def test_initial(nproc): datadir = Path("data") inputfile = datadir / "BOUT.inp" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) # Read the input file config = configparser.ConfigParser() diff --git a/tests/integrated/test-interchange-instability/test_interchange_instability.py b/tests/integrated/test-interchange-instability/test_interchange_instability.py index af0006c930..eb689d9913 100755 --- a/tests/integrated/test-interchange-instability/test_interchange_instability.py +++ b/tests/integrated/test-interchange-instability/test_interchange_instability.py @@ -4,8 +4,8 @@ # Run the test, compare results against the benchmark # import numpy as np -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell nproc = 2 # Number of processors to run on reltol = 1.0e-3 # Allowed relative tolerance in growth-rate @@ -16,12 +16,12 @@ def test_interchange_instability(): # Delete old output files - shell("rm data_1/BOUT.dmp.*") - shell("rm data_10/BOUT.dmp.*") + shell("rm -rf data_1/BOUT.dmp.*") + shell("rm -rf data_10/BOUT.dmp.*") def growth_rate(path, nproc, log=False): pipe = bool(log) - s, out = launch_safe( + _s, out = launch_safe( "./2fluid -d " + path, nproc=nproc, mthread=nthreads, pipe=pipe ) if pipe: @@ -54,12 +54,10 @@ def growth_rate(path, nproc, log=False): print(" Log file run_1.log") orig = 2.148177e05 # 24th October 2011, revision c4f7ec92786b333a5502c5256b5e602ba867090f analytic = 2.2e5 - print( - " Growth-rate = %e, original = %e, analytic = %e" % (growth, orig, analytic) - ) + print(f" Growth-rate = {growth:e}, original = {orig:e}, analytic = {analytic:e}") absdev = abs(growth - orig) reldev = absdev / orig - print(" Deviation from original: %e (%e %%)" % (absdev, reldev * 100.0)) + print(f" Deviation from original: {absdev:e} ({reldev * 100.0:e} %)") assert reldev <= reltol, " => Failed" @@ -71,11 +69,9 @@ def growth_rate(path, nproc, log=False): # orig = 65570. # 24th October 2011, revision c4f7ec92786b333a5502c5256b5e602ba867090f orig = 6.457835e04 # 25th April 2014, revision fd032da analytic = 6.3e4 - print( - " Growth-rate = %e, original = %e, analytic = %e" % (growth, orig, analytic) - ) + print(f" Growth-rate = {growth:e}, original = {orig:e}, analytic = {analytic:e}") absdev = abs(growth - orig) reldev = absdev / orig - print(" Deviation from original: %e (%e %%)" % (absdev, reldev * 100.0)) + print(f" Deviation from original: {absdev:e} ({reldev * 100.0:e} %)") assert reldev <= reltol, " => Failed" diff --git a/tests/integrated/test-interpolate-z/test_interpolate_z.py b/tests/integrated/test-interpolate-z/test_interpolate_z.py index 7c2052fc21..dc450e0793 100755 --- a/tests/integrated/test-interpolate-z/test_interpolate_z.py +++ b/tests/integrated/test-interpolate-z/test_interpolate_z.py @@ -4,11 +4,12 @@ # Run the test, compare results against the benchmark # -from boututils.run_wrapper import shell, launch_safe -from boutdata import collect -from numpy import sqrt, max, abs, mean, array, log, polyfit from sys import stdout +from boutdata import collect +from boututils.run_wrapper import launch_safe, shell +from numpy import abs, array, log, max, mean, polyfit, sqrt + # Display the plots as well as saving to file show_plot = False @@ -33,9 +34,9 @@ def test_interpolate_z(): print("Running ZInterpolation test") success = True - for method in methods: + for method, expected_order in methods.items(): print("------------------------------") - print("Using {} interpolation".format(method)) + print(f"Using {method} interpolation") error_2 = {} error_inf = {} @@ -52,10 +53,10 @@ def test_interpolate_z(): cmd = "./test_interpolate" + args - shell(["rm data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) - s, out = launch_safe(cmd, nproc=nproc, pipe=True) - with open("run.log.{}.{}".format(method, nx), "w") as f: + _s, out = launch_safe(cmd, nproc=nproc, pipe=True) + with open(f"run.log.{method}.{nx}", "w") as f: f.write(out) # Collect output data @@ -75,17 +76,17 @@ def test_interpolate_z(): error_2[var].append(l2) error_inf[var].append(linf) - print("{0:s} : l-2 {1:.8f} l-inf {2:.8f}".format(var, l2, linf)) + print(f"{var:s} : l-2 {l2:.8f} l-inf {linf:.8f}") dx = 1.0 / array(nxlist) for var in varlist: fit = polyfit(log(dx), log(error_2[var]), 1) order = fit[0] - stdout.write("{0:s} Convergence order = {1:.2f}".format(var, order)) + stdout.write(f"{var:s} Convergence order = {order:.2f}") # Make sure scaling is at least 90% of expected order - if order < 0.9 * methods[method]: + if order < 0.9 * expected_order: print("............ FAIL") success = False else: @@ -110,11 +111,11 @@ def test_interpolate_z(): plt.xlabel(r"Mesh spacing $\delta x$") plt.ylabel("Error norm") - plt.title("Error scaling for {}".format(method)) + plt.title(f"Error scaling for {method}") - name = "error_scaling_{}.pdf".format(method) + name = f"error_scaling_{method}.pdf" plt.savefig(name) - print("Plot saved to {}".format(name)) + print(f"Plot saved to {name}") if show_plot: plt.show() diff --git a/tests/integrated/test-interpolate/test_interpolate.py b/tests/integrated/test-interpolate/test_interpolate.py index 07a06a6c19..909a2d3c9b 100755 --- a/tests/integrated/test-interpolate/test_interpolate.py +++ b/tests/integrated/test-interpolate/test_interpolate.py @@ -4,13 +4,13 @@ # Run the test, compare results against the benchmark # -import pytest -from boututils.run_wrapper import shell, launch_safe -from boutdata import collect -import boutconfig -from numpy import sqrt, max, abs, mean, array, log, polyfit from sys import stdout +import boutconfig +import pytest +from boutdata import collect +from boututils.run_wrapper import launch_safe, shell +from numpy import abs, array, log, max, mean, polyfit, sqrt # Display the plots as well as saving to file show_plot = False @@ -39,7 +39,7 @@ def test_interpolate(method): success = True print("------------------------------") - print("Using {} interpolation".format(method)) + print(f"Using {method} interpolation") error_2 = {} error_inf = {} @@ -56,10 +56,10 @@ def test_interpolate(method): cmd = "./test_interpolate" + args - shell(["rm data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) - s, out = launch_safe(cmd, nproc=nproc, pipe=True) - with open("run.log.{}.{}".format(method, nx), "w") as f: + _s, out = launch_safe(cmd, nproc=nproc, pipe=True) + with open(f"run.log.{method}.{nx}", "w") as f: f.write(out) # Collect output data @@ -88,14 +88,14 @@ def myplot(f, lbl=None): error_2[var].append(l2) error_inf[var].append(linf) - print("{0:s} : l-2 {1:.8f} l-inf {2:.8f}".format(var, l2, linf)) + print(f"{var:s} : l-2 {l2:.8f} l-inf {linf:.8f}") dx = 1.0 / array(nxlist) for var in varlist: fit = polyfit(log(dx), log(error_2[var]), 1) order = fit[0] - stdout.write("{0:s} Convergence order = {1:.2f}".format(var, order)) + stdout.write(f"{var:s} Convergence order = {order:.2f}") # Make sure scaling is at least 90% of expected order if order < 0.9 * methods[method]: @@ -123,11 +123,11 @@ def myplot(f, lbl=None): plt.xlabel(r"Mesh spacing $\delta x$") plt.ylabel("Error norm") - plt.title("Error scaling for {}".format(method)) + plt.title(f"Error scaling for {method}") - name = "error_scaling_{}.pdf".format(method) + name = f"error_scaling_{method}.pdf" plt.savefig(name) - print("Plot saved to {}".format(name)) + print(f"Plot saved to {name}") if show_plot: plt.show() diff --git a/tests/integrated/test-invertable-operator/test_invertable_operator.py b/tests/integrated/test-invertable-operator/test_invertable_operator.py index f7b40e4204..f5ea660e98 100755 --- a/tests/integrated/test-invertable-operator/test_invertable_operator.py +++ b/tests/integrated/test-invertable-operator/test_invertable_operator.py @@ -4,8 +4,8 @@ # import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell nprocs = [1, 2] # Number of processors to run on reltol = 1.0e-3 # Allowed relative tolerance @@ -13,34 +13,29 @@ # Delete old output files -shell(["rm data/BOUT.dmp.*"]) +shell(["rm -rf data/BOUT.dmp.*"]) def run(path, nproc, log=False): pipe = bool(log) - s, out = launch_safe( + _s, out = launch_safe( "./invertable_operator -d " + path, nproc=nproc, mthread=nthreads, pipe=pipe ) if pipe: - f = open(log, "w") - f.write(out) - f.close() + with open(log, "w") as f: + f.write(out) # Get result of the test passVerification = collect("passVerification", path=path)[-1] maxRelErrLaplacians = collect("maxRelErrLaplacians", path=path)[-1] if passVerification == 0: - print( - " => Failed (verification step - value is {s})".format(s=passVerification) - ) + print(f" => Failed (verification step - value is {passVerification})") pytest.fail() if maxRelErrLaplacians > reltol: print( - " => Failed (relative tolerance step -- difference of {s})".format( - s=maxRelErrLaplacians - ) + f" => Failed (relative tolerance step -- difference of {maxRelErrLaplacians})" ) pytest.fail() diff --git a/tests/integrated/test-invpar/test_invpar.py b/tests/integrated/test-invpar/test_invpar.py index 6a911821be..ad62614800 100755 --- a/tests/integrated/test-invpar/test_invpar.py +++ b/tests/integrated/test-invpar/test_invpar.py @@ -4,19 +4,18 @@ # Run the test, check it completed successfully # -from boututils.run_wrapper import shell, launch - +from boututils.run_wrapper import launch, shell flags_src = [ - dict(acoef=1, bcoef=0, ccoef=0, dcoef=0, ecoef=0), - dict(acoef=1.5, bcoef="'2.*sin(2*y)'"), - dict(acoef=1), - dict(acoef=1, bcoef=2), - dict(ccoef=1.793), - dict(ccoef=3, bcoef=0), - dict(dcoef=3.5), - dict(ecoef=-1), - dict(input_field="'ballooning(exp(-y*y)*cos(z)*gauss(x,0.2))'"), + {"acoef": 1, "bcoef": 0, "ccoef": 0, "dcoef": 0, "ecoef": 0}, + {"acoef": 1.5, "bcoef": "'2.*sin(2*y)'"}, + {"acoef": 1}, + {"acoef": 1, "bcoef": 2}, + {"ccoef": 1.793}, + {"ccoef": 3, "bcoef": 0}, + {"dcoef": 3.5}, + {"ecoef": -1}, + {"input_field": "'ballooning(exp(-y*y)*cos(z)*gauss(x,0.2))'"}, ] @@ -36,10 +35,10 @@ def test_invpar(): for nproc in [1, 2, 4]: cmd = "./test_invpar" - print(" %d processors...." % (nproc)) + print(f" {nproc} processors....") r = 0 for f in flags: - shell(["rm data/BOUT.dmp.* 2> err.log"]) + shell(["rm -rf data/BOUT.dmp.* 2> err.log"]) # Run the case s, _ = launch(cmd + " -q -q -q " + f, nproc=nproc, mthread=1) diff --git a/tests/integrated/test-laplace/test_laplace.py b/tests/integrated/test-laplace/test_laplace.py index b2460c5d3a..d7fcc4cc30 100755 --- a/tests/integrated/test-laplace/test_laplace.py +++ b/tests/integrated/test-laplace/test_laplace.py @@ -1,15 +1,14 @@ #!/usr/bin/env python3 from pathlib import Path +import numpy.testing as npt + # # Run the test, compare results against the benchmark # - import pytest -import numpy.testing as npt -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect, create_cache - +from boututils.run_wrapper import launch_safe, shell # Variables to compare vars = [ @@ -51,9 +50,9 @@ def test_laplace(solver, nproc, benchmark_data): nxpe = 2 if nproc > 2 else 1 cmd = f"./test_laplace NXPE={nxpe} laplace:type={solver}" - shell("rm data/BOUT.dmp.*.nc") + shell("rm -rf data/BOUT.dmp.*") - s, out = launch_safe(cmd, nproc=nproc, mthread=1, pipe=True) + _s, out = launch_safe(cmd, nproc=nproc, mthread=1, pipe=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-laplacexy-fv/test_laplacexy_fv.py b/tests/integrated/test-laplacexy-fv/test_laplacexy_fv.py index 0e11b7be45..4b119b8e21 100755 --- a/tests/integrated/test-laplacexy-fv/test_laplacexy_fv.py +++ b/tests/integrated/test-laplacexy-fv/test_laplacexy_fv.py @@ -5,17 +5,21 @@ # import pytest -from boututils.run_wrapper import shell, launch, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch, launch_safe, shell tol = 5.0e-8 nproc = 8 argslist = [ - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' @@ -26,12 +30,18 @@ #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann', #'laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=neumann f:bndry_xout=neumann f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), ] @@ -41,7 +51,7 @@ def test_laplacexy_fv(args): print("Running LaplaceXY inversion test") cmd = f"./test-laplacexy {args}" - shell(["rm data/BOUT.dmp.*.nc > /dev/null 2>&1"]) + shell(["rm -rf data/BOUT.dmp.* > /dev/null 2>&1"]) s, out = launch( f"{cmd} laplacexy:pctype=hypre", nproc=nproc, pipe=True, verbose=True diff --git a/tests/integrated/test-laplacexy-short/test_laplacexy_short.py b/tests/integrated/test-laplacexy-short/test_laplacexy_short.py index d24ad14efd..eebf0323dd 100755 --- a/tests/integrated/test-laplacexy-short/test_laplacexy_short.py +++ b/tests/integrated/test-laplacexy-short/test_laplacexy_short.py @@ -6,19 +6,25 @@ import pytest -from boututils.run_wrapper import shell, launch, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch, launch_safe, shell tol = 5.0e-8 nproc = 8 argslist = [ - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3", + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3" + ), #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann ' @@ -29,18 +35,30 @@ #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann', #'laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=neumann f:bndry_xout=neumann f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1", + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1" + ), ] @@ -49,7 +67,7 @@ def test_laplacexy_short(args): cmd = f"./test-laplacexy {args}" - shell(["rm data/BOUT.dmp.*.nc > /dev/null 2>&1"]) + shell(["rm -rf data/BOUT.dmp.* > /dev/null 2>&1"]) s, out = launch( f"{cmd} laplacexy:pctype=hypre", nproc=nproc, pipe=True, verbose=True diff --git a/tests/integrated/test-laplacexy/test_laplacexy.py b/tests/integrated/test-laplacexy/test_laplacexy.py index fd32b9b223..2b1a6ac3ad 100755 --- a/tests/integrated/test-laplacexy/test_laplacexy.py +++ b/tests/integrated/test-laplacexy/test_laplacexy.py @@ -5,8 +5,8 @@ # import pytest -from boututils.run_wrapper import shell, launch, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch, launch_safe, shell tol_orth = 5.0e-8 @@ -17,12 +17,18 @@ tol_nonorth = 2.0e-5 argslist = [ - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3", + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=dirichlet " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3" + ), #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann ' @@ -33,18 +39,30 @@ #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann', #'laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=dirichlet ' #'f:bndry_xin=neumann f:bndry_xout=neumann f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann laplacexy:pctype=hypre", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 laplacexy:pctype=hypre", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1 laplacexy:pctype=hypre", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1 laplacexy:pctype=hypre", + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann laplacexy:pctype=hypre" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 laplacexy:pctype=hypre" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=free_o3 " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1 laplacexy:pctype=hypre" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=free_o3 " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=free_o3 f:bndry_ydown=free_o3 b:function=.1 laplacexy:pctype=hypre" + ), ] @@ -60,7 +78,7 @@ def test_laplacexy(is_nonorth, tol, case_idx, base_args): args = base_args if is_nonorth else f"{base_args} mesh:g12=0." cmd = f"./test-laplacexy {args}" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) if "hypre" in args: s, out = launch(cmd, nproc=nproc, pipe=True, verbose=True) diff --git a/tests/integrated/test-laplacexy2-hypre/test_laplacexy2_hypre.py b/tests/integrated/test-laplacexy2-hypre/test_laplacexy2_hypre.py index 189d364650..a60a26e541 100755 --- a/tests/integrated/test-laplacexy2-hypre/test_laplacexy2_hypre.py +++ b/tests/integrated/test-laplacexy2-hypre/test_laplacexy2_hypre.py @@ -12,10 +12,14 @@ nproc = 8 argslist = [ - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry_dirichlet=true " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry_dirichlet=true " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry_dirichlet=true ' #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', #'laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry_dirichlet=true ' @@ -26,25 +30,31 @@ #'f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann', #'laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry_dirichlet=true ' #'f:bndry_xin=neumann f:bndry_xout=neumann f:bndry_yup=dirichlet f:bndry_ydown=dirichlet', - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann", - "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " - "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", - "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " - "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1", + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann" + ), + ( + "laplacexy:core_bndry_dirichlet=true laplacexy:pf_bndry_dirichlet=true laplacexy:y_bndry=neumann " + "f:bndry_xin=dirichlet f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), + ( + "laplacexy:core_bndry_dirichlet=false laplacexy:pf_bndry_dirichlet=false laplacexy:y_bndry=neumann " + "f:bndry_xin=neumann f:bndry_xout=dirichlet f:bndry_yup=neumann f:bndry_ydown=neumann b:function=.1" + ), ] @pytest.mark.parametrize("args", argslist) def test_laplacexy2_hypre(args): - print(" %d processors...." % nproc) + print(f" {nproc} processors....") cmd = f"./test-laplacexy {args}" - shell(["rm data/BOUT.dmp.*.nc > /dev/null 2>&1"]) + shell(["rm -rf data/BOUT.dmp.* > /dev/null 2>&1"]) - s, out = launch_safe(cmd, nproc=nproc, pipe=True, verbose=True) + _s, out = launch_safe(cmd, nproc=nproc, pipe=True, verbose=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-laplacexz/test_laplacexz.py b/tests/integrated/test-laplacexz/test_laplacexz.py index 4262aa0ada..c10dbe3201 100755 --- a/tests/integrated/test-laplacexz/test_laplacexz.py +++ b/tests/integrated/test-laplacexz/test_laplacexz.py @@ -6,9 +6,8 @@ import numpy as np import pytest - from boutdata.collect import collect -from boututils.run_wrapper import shell, launch_safe, getmpirun +from boututils.run_wrapper import getmpirun, launch_safe, shell @pytest.mark.parametrize("nproc", [1, 2, 4]) @@ -23,10 +22,10 @@ def test_laplacexz(nproc): # Unique data directory per processor configuration cmd = f"./test-laplacexz nxpe={nproc}" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print(f" {nproc} processors (nxpe = {nproc})....") - s, out = launch_safe(cmd, runcmd=MPIRUN, nproc=nproc, mthread=1, pipe=True) + _s, out = launch_safe(cmd, runcmd=MPIRUN, nproc=nproc, mthread=1, pipe=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-multigrid_laplace/test_multigrid_laplace.py b/tests/integrated/test-multigrid_laplace/test_multigrid_laplace.py index d29a7596f3..074505c5ec 100755 --- a/tests/integrated/test-multigrid_laplace/test_multigrid_laplace.py +++ b/tests/integrated/test-multigrid_laplace/test_multigrid_laplace.py @@ -5,8 +5,8 @@ # import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell tol = 2e-6 # Absolute tolerance numTests = 4 # We test 4 different boundary conditions (with slightly different inputs for each) @@ -21,12 +21,12 @@ def test_multigrid_laplace(nproc, inputfile): # so splitting in y-direction is redundant (and also doesn't help test the multigrid solver) cmd = f"./test_multigrid_laplace -f {inputfile} NXPE={nproc} input:error_on_unused_options=false" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print("Running multigrid Laplacian inversion test") print(f"{nproc} processors, input file is: {inputfile}") - s, out = launch_safe(cmd, nproc=nproc, pipe=True) + _s, out = launch_safe(cmd, nproc=nproc, pipe=True) # Save a log file uniquely named for this specific combination input_file_name = inputfile.replace(".inp", "") diff --git a/tests/integrated/test-multigrid_laplace/test_multigrid_laplace_unsheared.py b/tests/integrated/test-multigrid_laplace/test_multigrid_laplace_unsheared.py index 627db84371..3e419b6cfa 100755 --- a/tests/integrated/test-multigrid_laplace/test_multigrid_laplace_unsheared.py +++ b/tests/integrated/test-multigrid_laplace/test_multigrid_laplace_unsheared.py @@ -5,8 +5,8 @@ # import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell tol = 1e-9 # Absolute tolerance numTests = 4 # We test 4 different boundary conditions (with slightly different inputs for each) @@ -23,13 +23,13 @@ def test_multigrid_laplace_unsheared(nproc): inputfile = "BOUT_unsheared.inp" cmd = f"./test_multigrid_laplace -f {inputfile} NXPE={nproc}" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print( f"Running unsheared multigrid Laplacian test: {nproc} procs, {mthread} thread(s)" ) - s, out = launch_safe(cmd, nproc=nproc, mthread=mthread, pipe=True) + _s, out = launch_safe(cmd, nproc=nproc, mthread=mthread, pipe=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-naulin-laplace/test_naulin_laplace.py b/tests/integrated/test-naulin-laplace/test_naulin_laplace.py index 32958c58de..1677dafcf3 100755 --- a/tests/integrated/test-naulin-laplace/test_naulin_laplace.py +++ b/tests/integrated/test-naulin-laplace/test_naulin_laplace.py @@ -1,8 +1,8 @@ #!/usr/bin/env python3 import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell tol = 2e-7 # Absolute tolerance numTests = 4 # We test 4 different boundary conditions (with slightly different inputs for each) @@ -17,10 +17,10 @@ def test_naulin_laplace(nproc): # set nxpe on the command line as we only use solution from one point in y, so splitting in y-direction is redundant (and also doesn't help test the solver) cmd = f"./test_naulin_laplace NXPE={nproc}" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print(f"Running LaplaceNaulin inversion test: {nproc} procs, {mthread} thread(s)") - s, out = launch_safe(cmd, nproc=nproc, mthread=mthread, pipe=True) + _s, out = launch_safe(cmd, nproc=nproc, mthread=mthread, pipe=True) with open(f"run.log.{nproc}", "w") as f: f.write(out) diff --git a/tests/integrated/test-petsc_laplace/test_petsc_laplace.py b/tests/integrated/test-petsc_laplace/test_petsc_laplace.py index 3a977d4f98..bf93563cdf 100755 --- a/tests/integrated/test-petsc_laplace/test_petsc_laplace.py +++ b/tests/integrated/test-petsc_laplace/test_petsc_laplace.py @@ -5,9 +5,10 @@ # import pathlib + import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect, create_cache +from boututils.run_wrapper import launch_safe, shell errors = [ "max_error1", @@ -24,10 +25,10 @@ def test_petsc_laplace(nproc): cmd = "./test_petsc_laplace" - shell(["rm data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) print(f" {nproc} processors....") - s, out = launch_safe(cmd, nproc=nproc, pipe=True, verbose=True) + _s, out = launch_safe(cmd, nproc=nproc, pipe=True, verbose=True) pathlib.Path(f"run.log.{nproc}").write_text(out) cache = create_cache(path="data", prefix="BOUT.dmp") diff --git a/tests/integrated/test-petsc_laplace_MAST-grid/test_petsc_laplace_MAST_grid.py b/tests/integrated/test-petsc_laplace_MAST-grid/test_petsc_laplace_MAST_grid.py index 1ecdc5a99a..63e5b1e852 100755 --- a/tests/integrated/test-petsc_laplace_MAST-grid/test_petsc_laplace_MAST_grid.py +++ b/tests/integrated/test-petsc_laplace_MAST-grid/test_petsc_laplace_MAST_grid.py @@ -5,8 +5,8 @@ # import pytest -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell # Variables to compare vars = [ @@ -31,9 +31,9 @@ def test_petsc_laplace_MAST_grid(nproc, jy): cmd = f"./test_petsc_laplace_MAST_grid mesh:file=grids/grid_MAST_SOL_jyis{jy}.nc" - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) - s, out = launch_safe(cmd, nproc=nproc, pipe=True) + _s, out = launch_safe(cmd, nproc=nproc, pipe=True) with open(f"run.log.{nproc}.jy_{jy}", "w") as f: f.write(out) diff --git a/tests/integrated/test-region-iterator/test_region_iterator.py b/tests/integrated/test-region-iterator/test_region_iterator.py index c7f84de6a4..76395903cf 100755 --- a/tests/integrated/test-region-iterator/test_region_iterator.py +++ b/tests/integrated/test-region-iterator/test_region_iterator.py @@ -4,8 +4,7 @@ # Run the test, check it completed successfully # -from boututils.run_wrapper import launch_safe - +from boututils.run_wrapper import launch_safe, shell flags = [""] cmd = "./test_region_iterator" @@ -17,15 +16,15 @@ def test_region_iterator(): for nproc in [1, 2]: # Number of mpi procs for mthread in [1]: # Number of omp threads (not yet supported) - print("\t{n} processors and {m} threads".format(n=nproc, m=mthread)) + print(f"\t{nproc} processors and {mthread} threads") for f in flags: # Run the case - s, out = launch_safe(cmd + " " + f, nproc=nproc, pipe=pipe) + shell(["rm -rf data/BOUT.dmp.*"]) + _s, out = launch_safe(cmd + " " + f, nproc=nproc, pipe=pipe) if pipe: - f = open("run.log." + str(nproc) + "." + str(mthread), "w") - f.write(out) - f.close() + with open(f"run.log.{nproc}.{mthread}", "w") as f: + f.write(out) # If we've got here we know that cmd launched by launch_safe passed # as otherwise it raises. diff --git a/tests/integrated/test-restart-io/test_restart_io.py b/tests/integrated/test-restart-io/test_restart_io.py index 2d5cb373c9..77d75437b0 100755 --- a/tests/integrated/test-restart-io/test_restart_io.py +++ b/tests/integrated/test-restart-io/test_restart_io.py @@ -3,14 +3,15 @@ # Test file I/O by loading from restart files and writing to dump files # +import uuid +from pathlib import Path + +import numpy from boutdata import restart from boutdata.collect import collect from boututils.boutarray import BoutArray from boututils.datafile import DataFile -from boututils.run_wrapper import shell, launch_safe -import numpy -import uuid -from pathlib import Path +from boututils.run_wrapper import launch_safe, shell nx = 8 ny = 16 @@ -93,25 +94,24 @@ def test_restart_io(): # y-processor-indices, and collect() cannot handle this. for nproc in [1, 2, 4]: # delete any existing output - shell("rm -f data/BOUT.dmp.*.nc data/BOUT.restart.*.nc") + shell("rm -rf data/BOUT.dmp.* data/BOUT.restart.*.nc") # create restart files for the run restart.redistribute(nproc, path=restartdir, output="data") - print(" %d processor...." % (nproc)) + print(f" {nproc} processor....") # run the test executable - s, out = launch_safe("./test-restart-io", nproc=nproc, pipe=True) + _s, out = launch_safe("./test-restart-io", nproc=nproc, pipe=True) with open("run.log." + str(nproc), "w") as f: f.write(out) # check the results - for name in testvars.keys(): + for name, testvar in testvars.items(): # check non-evolving version result = collect( name + "_once", path="data", xguards=True, yguards=True, info=False ) - testvar = testvars[name] if not numpy.allclose(testvar, result): success = False @@ -126,7 +126,7 @@ def test_restart_io(): if name == "fperp_lower" or name == "fperp_upper": yindex_result = result.attributes["yindex_global"] yindex_test = testvar.attributes["yindex_global"] - if not yindex_result == yindex_test: + if yindex_result != yindex_test: success = False print( "Fail: yindex_global of " @@ -149,7 +149,7 @@ def test_restart_io(): if name == "fperp_lower" or name == "fperp_upper": yindex_result = result.attributes["yindex_global"] yindex_test = testvar.attributes["yindex_global"] - if not yindex_result == yindex_test: + if yindex_result != yindex_test: success = False print( "Fail: yindex_global of " @@ -169,7 +169,7 @@ def test_restart_io(): success = False print(f"run_id='{run_id}' is not a valid UUID") run_restart_from = str(collect("run_restart_from", path="data", info=False)) - if not run_restart_from == run_id_string: + if run_restart_from != run_id_string: success = False print( f"incorrect run_restart_from='{run_restart_from}'. Expected '{run_id_string}'" @@ -179,5 +179,5 @@ def test_restart_io(): # clean up binary files shell( - "rm -f data/BOUT.dmp.*.nc data/BOUT.restart.*.nc data/restart/BOUT.restart.0.nc" + "rm -rf data/BOUT.dmp.* data/BOUT.restart.*.nc data/restart/BOUT.restart.0.nc" ) diff --git a/tests/integrated/test-restarting/test_restarting.py b/tests/integrated/test-restarting/test_restarting.py index b5f51bd073..d1262c2343 100755 --- a/tests/integrated/test-restarting/test_restarting.py +++ b/tests/integrated/test-restarting/test_restarting.py @@ -1,14 +1,16 @@ #!/usr/bin/env python3 import numpy as np -from boututils.run_wrapper import shell, launch_safe from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell def test_restarting(): + shell(["rm -rf data/BOUT.dmp.*"]) + # Run once for 10 timesteps - s, out = launch_safe("./test_restarting solver:nout=10", nproc=1, pipe=True) + _s, _out = launch_safe("./test_restarting solver:nout=10", nproc=1, pipe=True) # Read reference data f3d_0 = collect("f3d", path="data", info=False) @@ -19,9 +21,9 @@ def test_restarting(): print("-> Testing restart append") - shell(["rm -f data/BOUT.dmp.0.nc"]) - s, out = launch_safe("./test_restarting solver:nout=5", nproc=1, pipe=True) - s, out = launch_safe( + shell(["rm -rf data/BOUT.dmp.*"]) + _s, _out = launch_safe("./test_restarting solver:nout=5", nproc=1, pipe=True) + _s, _out = launch_safe( "./test_restarting solver:nout=5 restart append", nproc=1, pipe=True ) @@ -51,9 +53,11 @@ def test_restarting(): print("-> Testing restart") - shell(["rm -f data/BOUT.dmp.0.nc"]) - s, out = launch_safe("./test_restarting solver:nout=5", nproc=1, pipe=True) - s, out = launch_safe("./test_restarting solver:nout=5 restart", nproc=1, pipe=True) + shell(["rm -rf data/BOUT.dmp.*"]) + _s, _out = launch_safe("./test_restarting solver:nout=5", nproc=1, pipe=True) + _s, _out = launch_safe( + "./test_restarting solver:nout=5 restart", nproc=1, pipe=True + ) f3d_1 = collect("f3d", path="data", info=False) f2d_1 = collect("f2d", path="data", info=False) diff --git a/tests/integrated/test-smooth/test_smooth.py b/tests/integrated/test-smooth/test_smooth.py index 6c4fecc712..5c870055b5 100755 --- a/tests/integrated/test-smooth/test_smooth.py +++ b/tests/integrated/test-smooth/test_smooth.py @@ -4,12 +4,13 @@ # Run the test, compare results against the benchmark # -import pytest -import numpy as np import itertools from pathlib import Path -from boututils.run_wrapper import shell, launch_safe + +import numpy as np +import pytest from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell # Variables to compare vars = ["yavg2d", "yavg3d", "sm3d"] @@ -37,10 +38,10 @@ def test_smooth(benchmark_data, nxpe, nype): cmd = "./test_smooth" # Clean up old data - shell(["rm -f data/BOUT.dmp.*.nc"]) + shell(["rm -rf data/BOUT.dmp.*"]) # Run the executable - s, out = launch_safe(f"{cmd} NXPE={nxpe}", nproc=nproc, pipe=True) + _s, out = launch_safe(f"{cmd} NXPE={nxpe}", nproc=nproc, pipe=True) # Save log with open(f"run.log.{nproc}", "w") as f: diff --git a/tests/integrated/test-snes-save-jacobian/test_snes_save_jacobian.py b/tests/integrated/test-snes-save-jacobian/test_snes_save_jacobian.py index c524e791e5..a154bca0d7 100644 --- a/tests/integrated/test-snes-save-jacobian/test_snes_save_jacobian.py +++ b/tests/integrated/test-snes-save-jacobian/test_snes_save_jacobian.py @@ -1,3 +1,4 @@ +import subprocess from pathlib import Path import numpy as np @@ -6,7 +7,12 @@ TEST_DIR = Path(".") +def remove_bout_dmp_files(): + subprocess.run("rm -rf data/BOUT.dmp.*", shell=True, check=False) + + def test_runtest(assert_success_in_shell): + remove_bout_dmp_files() assert_success_in_shell("./test_snes_save_jacobian") data_dir = TEST_DIR / "data" @@ -71,6 +77,7 @@ def test_runtest(assert_success_in_shell): def test_rhs_jacobian(assert_success_in_shell): + remove_bout_dmp_files() assert_success_in_shell( "./test_snes_save_jacobian " "solver:jacobian_export_kind=rhs " diff --git a/tests/integrated/test-squash/test_squash.py b/tests/integrated/test-squash/test_squash.py index 58501be718..f55c4e77b4 100755 --- a/tests/integrated/test-squash/test_squash.py +++ b/tests/integrated/test-squash/test_squash.py @@ -1,19 +1,20 @@ #!/usr/bin/env python3 import itertools +import re import time +from pathlib import Path + import numpy as np -import re from boututils.datafile import DataFile from boututils.run_wrapper import launch_safe, shell_safe -from pathlib import Path IGNORED_VARS_PATTERN = re.compile( "(wtime|ncalls|arkode|cvode|run_id|run_restart_from|M.?SUB|N.?PE|iteration|wall_time|has_legacy_netcdf|hist_hi|openmp_threads).*" ) -class timer(object): +class timer: """Context manager for printing how long a command took""" def __init__(self, msg): @@ -24,7 +25,7 @@ def __enter__(self): def __exit__(self, exc_type, exc_value, traceback): end = time.time() - print("{:12.8f}s {}".format(end - self.start, self.msg)) + print(f"{end - self.start:12.8f}s {self.msg}") def timed_shell_safe(cmd, *args, **kwargs): @@ -41,17 +42,15 @@ def timed_launch_safe(cmd, *args, **kwargs): def verify(f1, f2): """Verifies that two BOUT++ files are identical""" - with timer("verify %s %s" % (f1, f2)): + with timer(f"verify {f1} {f2}"): d1 = DataFile(f1) d2 = DataFile(f2) - for v in d1.keys(): + for v in d1: if IGNORED_VARS_PATTERN.match(v): continue if d1[v].shape != d2[v].shape: - raise RuntimeError( - "shape mismatch in '{}': {} vs {}".format(v, d1[v], d2[v]) - ) + raise RuntimeError(f"shape mismatch in '{v}': {d1[v]} vs {d2[v]}") v1 = d1[v] v2 = d2[v] @@ -70,7 +69,7 @@ def verify(f1, f2): dimensions = [range(x) for x in v1.shape] for i in itertools.product(*dimensions): if v1[i] != v2[i]: - err += "{}: {} != {}\n".format(i, v1[i], v2[i]) + err += f"{i}: {v1[i]} != {v2[i]}\n" raise RuntimeError("data mismatch in ", v, err, v1, v2) @@ -81,39 +80,44 @@ def test_squash(): bout_squashoutput = "bout-squashoutput" print("Run once to get normal data") + timed_shell_safe("rm -rf data/BOUT.dmp.*") timed_shell_safe("./squash -q -q -q solver:nout=2") timed_shell_safe("mv data/BOUT.dmp.0.nc f1.nc") print("Parallel test") timed_shell_safe("rm -f f2.nc") + timed_shell_safe("rm -rf data/BOUT.dmp.*") timed_launch_safe("./squash -q -q -q solver:nout=2", nproc=4, mthread=1) - timed_shell_safe("{} -qdcl 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcl 9 data --outputname ../f2.nc") verify("f1.nc", "f2.nc") print("Parallel and in two pieces") timed_shell_safe("rm -f f2.nc") + timed_shell_safe("rm -rf data/BOUT.dmp.*") timed_launch_safe("./squash -q -q -q", nproc=4, mthread=1) - timed_shell_safe("{} -qdcl 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcl 9 data --outputname ../f2.nc") timed_launch_safe("./squash -q -q -q restart", nproc=4, mthread=1) - timed_shell_safe("{} -qdcal 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcal 9 data --outputname ../f2.nc") verify("f1.nc", "f2.nc") print("Parallel and in two pieces without dump_on_restart") timed_shell_safe("rm -f f2.nc") + timed_shell_safe("rm -rf data/BOUT.dmp.*") timed_launch_safe("./squash -q -q -q", nproc=4, mthread=1) - timed_shell_safe("{} -qdcl 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcl 9 data --outputname ../f2.nc") timed_launch_safe( "./squash -q -q -q restart dump_on_restart=false", nproc=4, mthread=1 ) - timed_shell_safe("{} -qdcal 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcal 9 data --outputname ../f2.nc") verify("f1.nc", "f2.nc") print("Sequential test") timed_shell_safe("rm -f f2.nc") + timed_shell_safe("rm -rf data/BOUT.dmp.*") timed_shell_safe("./squash -q -q -q solver:nout=2") - timed_shell_safe("{} -qdcl 9 data --outputname ../f2.nc".format(bout_squashoutput)) + timed_shell_safe(f"{bout_squashoutput} -qdcl 9 data --outputname ../f2.nc") verify("f1.nc", "f2.nc") diff --git a/tests/integrated/test-unused-options/test_unused_options.py b/tests/integrated/test-unused-options/test_unused_options.py index b147aa353c..69ea355ab9 100755 --- a/tests/integrated/test-unused-options/test_unused_options.py +++ b/tests/integrated/test-unused-options/test_unused_options.py @@ -1,13 +1,14 @@ #!/usr/bin/env python3 -from boututils.run_wrapper import launch_safe +from boututils.run_wrapper import launch_safe, shell def test_unused_options(): for nproc in [1, 2]: print(f"Running with nproc={nproc}...", end="") - s, out = launch_safe( + shell(["rm -rf data/BOUT.dmp.*"]) + _s, out = launch_safe( "./test_unused_options", nproc=nproc, mthread=1, diff --git a/tests/integrated/test-yupdown-weights/test_yupdown_weights.py b/tests/integrated/test-yupdown-weights/test_yupdown_weights.py index ae8754d50d..d1382ef86f 100755 --- a/tests/integrated/test-yupdown-weights/test_yupdown_weights.py +++ b/tests/integrated/test-yupdown-weights/test_yupdown_weights.py @@ -1,17 +1,18 @@ #!/usr/bin/env python3 -from boututils.run_wrapper import launch_safe -from boutdata.collect import collect from sys import stdout -from numpy import max, abs +from boutdata.collect import collect +from boututils.run_wrapper import launch_safe, shell +from numpy import abs, max def test_yupdown_weights(): failed = False for shifttype in ["shiftedinterp"]: - s, out = launch_safe( + shell(["rm -rf data/BOUT.dmp.*"]) + _s, out = launch_safe( "./test_yupdown_weights mesh:paralleltransform:type=" + shifttype, nproc=1, pipe=True, @@ -23,16 +24,16 @@ def test_yupdown_weights(): vars = [("ddy", "ddy2")] for v1, v2 in vars: - stdout.write("Testing %s and %s ... " % (v1, v2)) + stdout.write(f"Testing {v1} and {v2} ... ") ddy = collect(v1, path="data", xguards=False, yguards=False, info=False) ddy2 = collect(v2, path="data", xguards=False, yguards=False, info=False) diff = max(abs(ddy - ddy2)) if diff < 1e-8: - print(shifttype + " passed (Max difference %e)" % (diff)) + print(shifttype + f" passed (Max difference {diff:e})") else: - print(shifttype + " failed (Max difference %e)" % (diff)) + print(shifttype + f" failed (Max difference {diff:e})") failed = True assert not failed diff --git a/tests/integrated/test-yupdown/test_yupdown.py b/tests/integrated/test-yupdown/test_yupdown.py index 6d9f5de9e4..693cdc7a3a 100755 --- a/tests/integrated/test-yupdown/test_yupdown.py +++ b/tests/integrated/test-yupdown/test_yupdown.py @@ -1,9 +1,10 @@ #!/usr/bin/env python3 import os + import pytest -from boututils.run_wrapper import launch_safe from boutdata.collect import collect -from numpy import max, abs +from boututils.run_wrapper import launch_safe, shell +from numpy import abs, max shift_types = ["shifted", "shiftedinterp"] @@ -13,7 +14,9 @@ def test_case(shift_type): os.environ["OMPI_MCA_rmaps_base_oversubscribe"] = "1" # MPI oversubscribe - s, out = launch_safe( + shell(["rm -rf data/BOUT.dmp.*"]) + + _s, out = launch_safe( "./test_yupdown mesh:paralleltransform:type=" + shift_type, nproc=1, pipe=True, @@ -25,7 +28,7 @@ def test_case(shift_type): errors = [] for v, v_check in [("ddy", "ddy_check"), ("ddy2", "ddy_check")]: - print("Testing %s and %s ... " % (v, v_check)) + print(f"Testing {v} and {v_check} ... ") ddy = collect(v, path="data", xguards=False, yguards=False, info=False) ddy_check = collect( v_check, path="data", xguards=False, yguards=False, info=False