Skip to content

some errors when running with GPU #63

Closed

Description

@zgq-gg

We encountered some errors when running with GPU. Here are the results of the terminal output:

Image

This is my GPU compile instruction:
cmake -B build -DENABLE_MPI=ON -DENABLE_FORTRAN=OFF -D BLT_CXX_STD=c++17 -DCALIPER_DIR=/usr/local -DMFEM_DIR=/usr/local -DECMECH_DIR=/usr/local -DRAJA_DIR=/usr/local/lib/cmake/raja -D ENABLE_CUDA=ON -DENABLE_SNLS_V03=ON -DCMAKE_CUDA_ARCHITECTURES=70 -D PYTHON_EXECUTABLE=/usr/bin/python3.10 -DSNLS_DIR=/usr/local -D HYPRE_DIR=/usr/local/lib/cmake/HYPRE -D UMPIRE_DIR=/usr/local/lib/cmake/umpire -DCAMP_DIR=/usr/local/lib/cmake/camp -D ENABLE_TESTS=OFF -D BUILD_GTEST=OFF -D BUILD_TESTING=OFF -D ENABLE_GTEST=OFF -D ENABLE_OPENMP=ON -DCMAKE_EXE_LINKER_FLAGS="-L/usr/local/openmpi/lib -lmpi -lgomp -lgfortran" -D CMAKE_C_COMPILER=/bin/gcc -D CMAKE_CXX_COMPILER=/bin/g++ -D MPI_Fortran_COMPILER=/usr/local/openmpi/bin/mpifort -D CMAKE_CXX_FLAGS="-fPIC -std=c++17" -D CMAKE_C_FLAGS="-fPIC" -D CMAKE_CUDA_FLAGS="-restrict --expt-extended-lambda --expt-relaxed-constexpr -Xcompiler=-fPIC"

as well as the files and scripts I run

Image Image Image

element_region.txt
grain.txt
orientations.txt
props.txt
region_mapping.txt
regions.txt
state.txt

Activity

  1. rcarson3 commented on May 23, 2026

    @rcarson3
    Member

    @zgq-gg as it looks like you've just got a single phase material the region mapping files aren't required. The grain file also isn't needed for non-auto generated meshes that information should already be defined in the mesh as the PhysicalNames section if you're passing in GMSH . If it's a native MFEM mesh than it would already be defined by the element attributes. Next are you actually building things for the nvidia V100 GPUs? If not you need to make sure the GPU ID is the correct one for your GPUs. Updating that will require a full rebuild of all the packages though to the correct value.

    Outside of that, a few small things: the visualization files have a bug in the current release that output them every step and the post-processing steps are no longer every step. This is a bug being fixed in the #61 PR.

    Next, I noticed another bug recently that ExaCMech models and the orientation info is no longer automatically working out where it should go. So, I would update you [Material.Grain] sub-block to something like the below. I'll fix that in an upcoming release. It's possible this will fix your issue. If it doesn't I don't have the bandwidth to help run some of these issues down as GPU issues can be complicated or try to work out potential build issues. My recommendation if this doesn't fix things just throw a coding agent at it and see if it can't fix it locally if you don't have experience coding or debugging GPU programs.

        [Materials.Grain]
            orientation_file  = FILE_NAME
            ori_type          = "quat"
            ori_stride        = 4
            # This needs to be 9 for ExaCMech files for now
            ori_state_var_loc = 9
            num_grains        = ####
    

    Finally just for obvious reasons, I'm not going to download random untrusted files off the web.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions