Java MySQL bulk insert vs normal -
i tried make insert mysql faster, werote code bulk insert. problem when run following code, results same (time elapsed: 2min, or 173sec) both methods. comment on matter welcome.
mysqlconn mysqlconn = new mysqlconn(db,username,pass); connection conn =mysqlconn.getconnection(); preparedstatement ps = null; string query = "insert table (column) values (?)"; system.out.println("bulk insert started "+10000 ); long starttime = system.currenttimemillis(); try { ps = conn.preparestatement(query); long start = system.currenttimemillis(); for(int =0; i<10000;i++){ ps.setstring(1, "name"+i); ps.addbatch(); if(i%1000 == 0) ps.executebatch(); } ps.executebatch(); system.out.println("time taken="+(system.currenttimemillis()-start)); } catch (sqlexception e) { e.printstacktrace(); } system.out.println("end inserting data..."); long stoptime = system.currenttimemillis(); long elapsedtime = (stoptime - starttime); long days = timeunit.milliseconds.todays(elapsedtime); long hours = timeunit.milliseconds.tohours(elapsedtime); long minutes = timeunit.milliseconds.tominutes(elapsedtime); long seconds = timeunit.milliseconds.toseconds(elapsedtime); system.out.println("time elapsed: "+minutes+"min, or "+seconds+"sec"); system.out.println("normal insert started "+10000 ); long starttimenormal = system.currenttimemillis(); for(int i=0;i<10000;i++){ string ins="data"+i; insert insert = new insert(conn,ins,"db.table", "column" ); } system.out.println("end inserting data..."); long stoptimenormal = system.currenttimemillis(); long elapsedtimenormal = (stoptimenormal - starttimenormal); long daysn = timeunit.milliseconds.todays(elapsedtime); long hoursn = timeunit.milliseconds.tohours(elapsedtime); long minutesn = timeunit.milliseconds.tominutes(elapsedtime); long secondsn = timeunit.milliseconds.toseconds(elapsedtime); system.out.println("time elapsed: "+minutesn+"min, or "+secondsn+"sec");
you use addbatch
, executebatch
, good, need set autocommit on connection false in order achieve faster execution times.
something along lines:
connection conn = mysqlconn.getconnection(); conn.setautocommit(false); // here preparedstatement ps = null; string query = "insert table (column) values (?)"; system.out.println("bulk insert started "+10000 ); long starttime = system.currenttimemillis(); try { ps = conn.preparestatement(query); long start = system.currenttimemillis(); for(int =0; i<10000;i++){ ps.setstring(1, "name"+i); ps.addbatch(); if(i%1000 == 0) { ps.executebatch(); conn.commit(); // here } } ps.executebatch(); conn.commit(); // here system.out.println("time taken="+(system.currenttimemillis()-start)); } catch (sqlexception e) { e.printstacktrace(); }
Comments
Post a Comment