Performance between BufferedWriter and OutputStramWriter

Question

My requirement is to fetch almost 0.5 million records from the db2 database using Java.These fetched records I am going to write in text file. Currently I preferred BufferedWriter to do this as below -

Considering performance which includes execution time,faster I/O operations and less memory usage Which option I should go for?.

Test1 - Where I fetched almost 3200 records from database BufferedWriter - takes 9 sec OutputStramWriter - takes 8 sec

But I am confused between those as I have to fetch almost 0.5 million records in single shot where performance is the key factor!!.

try { Statement stmt = new Statement(); Connection con = getConnection(); BufferedWriter bw = getWriter(); String query = "select field1,field2,field3,field4,field5 from table1 WITH UR"; stmt = con.createStatement(); stmt.setFetchSize(10000); ResultSet rs = stmt.executeQuery(query); StringBuffer sb = new StringBuffer(); while (rs.next()) { sb.append(rs.getString(field1)); sb.append(rs.getString(field2)); sb.append(rs.getString(field3)); sb.append(rs.getString(field4)); sb.append(rs.getString(field5)); bw.write(sb.toString()); bw.newLine(); sb.setLength(0); } bw.flush(); } catch(SQLException ex) { ex.printstackTrace(); } catch(Exception ex2) { ex2.printstackTrace(); } finally() { stmt.close(); con.close(); bw.close(); } protected static BufferedWriter/OutputStreamWriter getWriter() { //here I have tow options //1. //FileOutputStream fos = new FileOutputStream(file); //OutputStreamWriter osr= new OutputStreamWriter(fos); // return osr; //2. //BufferedWriter out = new BufferedWriter(new FileWriter(file)); //return out; }

"Which option I should go for?" Testing the results in your real-world conditions and choosing the one that gives you the best results. — T.J. Crowder
– T.J. Crowder, Commented Jul 12, 2015 at 6:58
Thanks T.J.Crowder for your reply. But in my local m/c I don't have that much bunch of records.I do have only 3200 records based on that I have to choose between those. — vishal_k
– vishal_k, Commented Jul 12, 2015 at 7:02
Records can be created. Or you can do repeated passes through the 3200 you have. — T.J. Crowder
– T.J. Crowder, Commented Jul 12, 2015 at 7:06
I would caution against optimization if you are not certain of what the performance is against even 50% of the subset. I would encourage profiling before doing any idle speculation about which way is faster. — Makoto
– Makoto, Commented Jul 12, 2015 at 7:07
Thanks T.J.Crowder, I will go for repeated passes to assess the performance. — vishal_k
– vishal_k, Commented Jul 12, 2015 at 7:14

JB Nizet · Accepted Answer · 2015-07-12 07:20:55Z

Option 1: OutputStreamWriter writes to FileOutputStream
Option 2: BufferedWriter writes to FileWriter, which writes to FileOutputStream

Option 1 and Option 2 both use a buffer. Option 1 allows choosing a character encoding, whereas option 2 doesn't.

So the performance should be similar in both cases, but I would choose option 1, because choosing the appropriate encoding is essential. The performance is probably, anyway, largely directed by the database, and less so by the file IO. You'll never know that until you test in real conditions. For example, using a local database instead of a database on the network could change everything. The performance of the disk is also crucial.

What I would do though is avoiding the useless StringBuffer. You should write directly to the writer. StringBuilder, BTW, should be preferred to StringBuffer, as it doesn't have useless synchronization penalty.

Collectives™ on Stack Overflow

Performance between BufferedWriter and OutputStramWriter

1 Answer 1

Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Related