c++ - Communication cost of MPI send and receive -
i'm new mpi , want measure communication cost of mpi_send , mpi_recv between 2 nodes. have written following code purpose:
/*============================================================== * print_elapsed (prints timing statistics) *==============================================================*/ void print_elapsed(const char* desc, struct timeval* start, struct timeval* end, int numiterations) { struct timeval elapsed; /* calculate elapsed time */ if(start->tv_usec > end->tv_usec) { end->tv_usec += 1000000; end->tv_sec--; } elapsed.tv_usec = end->tv_usec - start->tv_usec; elapsed.tv_sec = end->tv_sec - start->tv_sec; printf("\n%s total elapsed time = %ld (usec)\n", desc, (elapsed.tv_sec*1000000 + elapsed.tv_usec)/numiterations ); } int main(int argc, char **argv) { int nprocs, nelements; /* command line args */ int my_id; long double* buffer, *rec_buffer; /* gettimeofday stuff */ struct timeval start, end; /* gettimeofday stuff */ struct timezone tzp; mpi_status status; /* status variable mpi operations */ mpi_init(&argc, &argv); mpi_comm_rank(mpi_comm_world, &my_id); /* getting id process */ /*--------------------------------------------------------- * read command line * - check usage , parse args *---------------------------------------------------------*/ if(argc < 2) { if(my_id == 0) printf("usage: %s [nelements]\n\n", argv[0]); mpi_finalize(); exit(1); } nelements = atoi(argv[1]); int numiterations = 64; mpi_comm_size(mpi_comm_world, &nprocs); /* number of processors */ if(my_id == 0) printf("\nexecuting %s: numelements=%d \n", argv[0], nelements); buffer = (long double *) malloc(sizeof(long double)*nelements); rec_buffer = (long double *) malloc(sizeof(long double)*nelements); if(buffer == null) { printf("processor %d - unable malloc()\n", my_id); mpi_finalize(); exit(1); } mpi_barrier(mpi_comm_world); if(my_id == 1) gettimeofday(&start, &tzp); for(int = 0 ; < numiterations ; ++i) { if(my_id == 0) mpi_send(buffer, nelements, mpi_long, 1, 0, mpi_comm_world); if(my_id == 1) mpi_recv(rec_buffer, nelements, mpi_long, 0, 0, mpi_comm_world, &status); } if(my_id == 1) { gettimeofday(&end,&tzp); } mpi_barrier(mpi_comm_world); if(my_id == 1) { print_elapsed("summation", &start, &end, numiterations); } free(buffer); mpi_finalize(); return 0; } /* main() */
i repeat send , receive numiteration
times, have no information initialization cost , real communication time. wondering if there better methods or tools measure communication cost more detail.
if want level of detail describe, you'll have down implementation of mpi library itself. you're measuring impact of communication on application. however, it's possible there's more involved in communication depending on infrastructure. networks can make progress without involving application , mpi libraries can use thread asynchronously make progress on messages well.
how measure these things depend on system , above constraints. if care how time application spending blocking in communication calls, more or less have accomplished that. use other tracing tools accomplish similar things (hpctoolkit 1 comes mind i've used in past).
if want more detailed information what's going on under hood, you'll have poke around inside implementation , start instrumenting internally (assuming you're using open source implementation such mpich or open mpi). more involved process , mechanisms change 1 implementation another.
Comments
Post a Comment