c++ - OpenMPI: MPI_Send allocates more and more memory -

i'm working on code crashes after time during mpi communication due running out of memory. wondering why code uses more , more memory since tried reduce allocations as possible. college of mine has similar problem code.

so found seems caused send , recv routine , therefore wrote example shows similar behavior (in smaller scale):

#include <mpi.h> #include <stdio.h> #include <sys/resource.h> #include <iomanip>  rusage mem_usage; int mem_proc; int mem_max; int mem_min; int mem_sum; int mem_sum_old; double mem_avg;  int world_size; int world_rank;   void printmemusage(std::string prefix) {     mpi_barrier(mpi_comm_world);      getrusage( rusage_self, &mem_usage);      mem_proc = mem_usage.ru_maxrss;      mpi_allreduce(&mem_proc,&mem_sum,1,mpi_int,mpi_sum,mpi_comm_world);     mpi_allreduce(&mem_proc,&mem_min,1,mpi_int,mpi_min,mpi_comm_world);     mpi_allreduce(&mem_proc,&mem_max,1,mpi_int,mpi_max,mpi_comm_world);       mem_avg = ((double) mem_sum) / ((double) world_size);      if(mem_sum_old == 0)         mem_sum_old = mem_sum;      if(world_rank == 0) std::cout << "("<< world_rank << ") " << std::setw(50) << std::left << prefix << " mem: " << mem_sum << " kb.  min: " << mem_min << " max: " << mem_max << " avg: " << mem_avg << " change: " << (mem_sum)-(mem_sum_old) << " kb" << std::endl;     mem_sum_old = mem_sum; }   int main(int argc, char** argv) {     mpi_init(null, null);     mpi_comm_size(mpi_comm_world, &world_size);     mpi_comm_rank(mpi_comm_world, &world_rank);      double send = 42.0;     double recv = 1;     int i;     mpi_request request[2];      printmemusage("start");      for(i = 0; < 10000; ++i)     {         mpi_send (&send, 1, mpi_double, (world_rank+1)%world_size, i, mpi_comm_world);         mpi_recv (&recv, 1, mpi_double, (world_rank-1)%world_size, i, mpi_comm_world, mpi_status_ignore);                }      printmemusage("end");      mpi_finalize(); } 

running code 2 cores leads following output:

mpirun -np 2 ./test (0) start                                              mem: 53068 kb.  min: 26528 max: 26540 avg: 26534 change: 0 kb (0) end                                                mem: 53300 kb.  min: 26632 max: 26668 avg: 26650 change: 232 kb 

the needed memory rises more cores involved. therefore seems send , recv functions allocates more , more memory. putting output loop shows additional memory not allocated once, allocated several times (but not always).

i have tested several mpi , gcc versions example gcc 4.9.2 , mpi 1.8.3 amount of needed additional memory seems vary chosen version.

does know why happening , how can work around this?

thank , best wishes

klaus