removing redundant line from a file

Question

here it is, using this function:

 private static void write(String Swrite) throws IOException { if (!StopWordRemoval.exists()) { StopWordRemoval.createNewFile(); } FileOutputStream fop = new FileOutputStream(file); if (Swrite != null) fop.write(Swrite.getBytes()); fop.flush(); fop.close(); }

my program gets string from user and write it into a file. after all users done with inputting their info, i want to remove the redundant info. if two exact lines, then one removes. first i tried the below codes but didnt worke out:

 private static void Normalize(File file) throws FileNotFoundException, IOException { String tempLine2; BufferedReader buf = new BufferedReader(new FileReader(file)); FileOutputStream fop = new FileOutputStream(temp, true); String tempLine = null; tempLine = buf.readLine(); fop.write(tempLine.getBytes()); BufferedReader buf2 = new BufferedReader(new FileReader(temp)); while ((tempLine = buf.readLine()) != null) { while ((tempLine2 = buf2.readLine()) != null) { if (!tempLine.trim().equals(tempLine2)) { if (tempLine != null) for (final String s : tempLine.split(" ")) { fop.write(s.getBytes()); fop.write(System.getProperty("line.separator").getBytes()); } } } } }

my idea in the second function was as below: writing first line into a new file, then comparing second line with it, if different then write, then comparing third line with both...but it seems my function sucks. any help?

Please care to format your code properly. A neat format is required to look it up. — Lion
– Lion, Commented Dec 16, 2011 at 12:12
@Lion his\her dry humor :) i tried to format it, hope it looks better now — lonesome
– lonesome, Commented Dec 16, 2011 at 12:16

yatskevich · Accepted Answer · 2011-12-16 12:13:06Z

3

Create a Set of lines. Consider this pseudo-code:

Set<String> uniqueLines = new HashSet<String>(); String line = readLine(); if (!uniqueLines.contains(line)) { write_to_file(line); uniqueLines.add(line); }

answered Dec 16, 2011 at 12:13

yatskevich

2,09316 silver badges26 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

lonesome Over a year ago

you mean first i fill the hashset by redundant file?

yatskevich Over a year ago

HashSet will contain lines you've already written to a new file. A line will appear in the new file if it differs from all other lines.

lonesome Over a year ago

oh i got it, first read first line, then compare with the hashset, if not same,then write it to the file and add to the hashset and go on....

yatskevich Over a year ago

right! you can even don't treat the first line as special one - the first line will be added for sure because HashSet will be empty at the moment :)

Jigar Joshi · Accepted Answer · 2011-12-16 12:11:19Z

2

Just read file line by line into a Set and at the end write new file from data from Set

answered Dec 16, 2011 at 12:11

Jigar Joshi

241k42 gold badges409 silver badges446 bronze badges

Comments

st0le · Accepted Answer · 2011-12-16 12:40:07Z

Ok, your approach can be better. I think this might be homework, so I'm not going to post any code...

For the Normalize function,

Open the file
Have a Set<String> declared and initialized (TreeSet will get you sorted results)
Read everyline and add it into the Set
Overwrite that file with the entries of the Set as each line.

(Explanation: Close the FileInputStream, and Create a new PrintStream(sameFile); which will essentially delete the previous contents and then start out.println(eachLine), finally close the file)
Done.

Collectives™ on Stack Overflow

removing redundant line from a file

3 Answers 3

4 Comments

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

4 Comments

Comments

Comments

Related