libmxm is a Mellanox library. It is likely that the MPI library used at your cluster is built with libmxm, however the runtime configuration you are using is not finding it.
Check you cluster documentation on how to run MPI jobs (may need something like module load mpi in your batch scripts). Failing that, contact the person responsible for your cluster, describe the problem and ask how to run MPI jobs with the proper environment.
Apparently, there has been some issue with the MPI library, so a representative from my school server suggested not to do use MPI (i.e. gmx instead of gmx_mpi), which solved the issue.