hadoop - Apache PIG - How can I display the execution time of a script? -
is there possibility show whole execution time of pig script? best thing generate separate file contains total time (e.g. hh:mm:s) taken execute script.
please give me pig code example realize this.
timing udfs
the first step improving performance , efficiency measuring time going. pig provides light-weight method approximately measuring how time spent in different user-defined functions (udfs) , loaders. set pig.udf.profile property true. cause new counters tracked map-reduce jobs generated script: approx_microsecs measures approximate amount of time spent in udf, , approx_invocations measures approximate number of times udf invoked. note may produce large number of counters (two per udf). excessive amounts of counters can lead poor jobtracker performance, use feature carefully, , preferably on test cluster.
you can view job counters in resource manager web ui.
Comments
Post a Comment