hadoop - Apache PIG - How can I display the execution time of a script? -


is there possibility show whole execution time of pig script? best thing generate separate file contains total time (e.g. hh:mm:s) taken execute script.

please give me pig code example realize this.

timing udfs

the first step improving performance , efficiency measuring time going. pig provides light-weight method approximately measuring how time spent in different user-defined functions (udfs) , loaders. set pig.udf.profile property true. cause new counters tracked map-reduce jobs generated script: approx_microsecs measures approximate amount of time spent in udf, , approx_invocations measures approximate number of times udf invoked. note may produce large number of counters (two per udf). excessive amounts of counters can lead poor jobtracker performance, use feature carefully, , preferably on test cluster.

you can view job counters in resource manager web ui.

from - https://pig.apache.org/docs/r0.11.1/perf.html


Comments

Popular posts from this blog

php - failed to open stream: HTTP request failed! HTTP/1.0 400 Bad Request -

java - How to filter a backspace keyboard input -

java - Show Soft Keyboard when EditText Appears -