Reduce performance time

Question

 'Check if the file has any broken records Dim reader As StreamReader = New StreamReader(fileDirectory) Dim fileLine As String Dim stopCheck As Boolean = False Do While reader.Peek() > -1 fileLine = reader.ReadLine() 'If the line is not start with eCW| then it is a broken record If Not fileLine.StartsWith("eCW|") Then stopCheck = True Exit Do End If Loop reader.Close() If stopCheck Then

It will take a very long time to validate the rows when the text file has many records.

ex. File 1 has 500,000 completed records. It will looping through all the rows until it close the program.

File 2 has a broken records toward the end of text file. It will have to looping through all the rows before it find the broken record.

Is there a way to speed up this validation process?

This method is already very quick (have you measured the time requirements under the conditions you are proposing?). The problem with quicker methods is that they need the whole file to be loaded in memory what is not possible with too big files. On the other hand, when dealing with so many records, relying on files doesn't seem the best approach (databases are precisely meant for this kind of situations). Lastly, bear in mind that you have to dispose the StreamReader variable to avoid problems (e.g., file getting locked); you can easily take care of that with the Using statement. — user2480047
– user2480047, Commented Oct 28, 2015 at 15:11

JerryM · Accepted Answer · 2015-10-28 15:49:43Z

Eliminating the use of the fileLine variable will eliminate a memory allocation on each iteration.

 Do While reader.Peek() > -1 'If the line is not start with eCW| then it is a broken record If Not reader.ReadLine().StartsWith("eCW|") Then stopCheck = True Exit Do End If Loop

Since strings are immutable, a memory allocation takes place each time the string is changed. Link Other than this, I think your approach is great--once you add the Using statement that is.

Rob · Accepted Answer · 2015-10-29 01:57:53Z

Just winging it here but ...

 try while not reader.readline().startswith("eCWL") end while stopCheck = true catch stopCheck = false end try

I'm guessing the catching the read exception when you try to read beyond eof will be well worth not needing to do the peek and the read for each record

Collectives™ on Stack Overflow

Reduce performance time

2 Answers 2

Comments

Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

Comments

Comments

Related