Simulating String.split using StringTokenizer

Question

I'm converting code from an existing application to compile against a Java 1.1 compiler for a custom piece of hardware. This means that I can't use String.split(regex) to convert my existing string into an array.

I created a method which should give the same result as String.split(regex) but there's something wrong with it and I can't figure out what.

Code:

private static String[] split(String delim, String line) { StringTokenizer tokens = new StringTokenizer(line, delim, true); String previous = ""; Vector v = new Vector(); while(tokens.hasMoreTokens()) { String token = tokens.nextToken(); if(!",".equals(token)) { v.add(token); } else if(",".equals(previous)) { v.add(""); } else { previous = token; } } return (String[]) v.toArray(new String[v.size()]); }

Sample input:

RM^RES,0013A2004081937F,,9060,1234FF

Sample output:

String line = "RM^RES,0013A2004081937F,,9060,1234FF"; String[] items = split(",", line); for(String s : items) { System.out.println(" [ " + s + " ] "); }

[ RM^RES ] [ 0013A2004081937F ] [ ] [ ] [ 9060 ] [ ] [ 1234FF ]

Desired output:

[ RM^RES ] [ 0013A2004081937F ] [ ] [ 9060 ] [ 1234FF ]

Old code that I'm trying to convert:

String line = "RM^RES,0013A2004081937F,,9060,1234FF"; String[] items = line.split(","); for(String s : items) { System.out.println(" [ " + s + " ] "); }

[ RM^RES ] [ 0013A2004081937F ] [ ] [ 9060 ] [ 1234FF ]

You probably want to replace the ",".equals(...) with delim.equals(...) in your split method, if you ever plan to use another delimiter. — Harald K
– Harald K, Commented Jul 11, 2013 at 9:00
@haraldK Thank you, that's a really good remark to make aside the problem :) — JREN
– JREN, Commented Jul 11, 2013 at 9:03

Andrei Micu · Accepted Answer · 2013-07-11 09:11:17Z

I modified the code and tested it. It works (don't forget to avoid hard-coding the "," so you can use the function for any delimiter):

private static String[] split(String delim, String line) { StringTokenizer tokens = new StringTokenizer(line, delim, true); String previous = delim; Vector v = new Vector(); while (tokens.hasMoreTokens()) { String token = tokens.nextToken(); if (!delim.equals(token)) { v.add(token); } else if (previous.equals(delim)) { v.add(""); } previous = token; } return (String[]) v.toArray(new String[v.size()]); }

JREN · Accepted Answer · 2013-07-11 09:32:16Z

1

Almost everything is right. Almost, because you forget to "clear" value of previous. Try this:

if(!",".equals(token)) { v.add(token); previous = ""; } else if(",".equals(previous)) { v.add(""); previous = ""; } else { previous = token; }

edited Jul 11, 2013 at 9:32

JREN

3,6204 gold badges29 silver badges46 bronze badges

answered Jul 11, 2013 at 8:53

Max Adamek

714 bronze badges

3 Comments

JREN Over a year ago

This still gives me incorrect output. [ RM^RES ] [ 0013A2004081937F ] [ ] [ 9060 ] [ ] [ 1234FF ]

Max Adamek Over a year ago

try to put previous = token in if{} too

JREN Over a year ago

It works after that, but Andrei M already gave a better solution. Thanks anyway.

mavroprovato · Accepted Answer · 2013-07-11 09:00:25Z

How about not using StringTokenizer at all:

private static String[] split(String delim, String line) { String current = line; int index = line.indexOf(delim); Vector vector = new Vector(); while (index != -1) { vector.add(current.substring(0, index)); current = current.substring(index + 1); index = current.indexOf(delim); } vector.add(current); return (String[]) vector.toArray(new String[vector.size()]); }

Ruchira Gayan Ranaweera · Accepted Answer · 2013-07-11 09:01:08Z

You can try it this way

 public static void main(String[] args) throws ParseException { for (String s : split(",", "RM^RES,0013A2004081937F, ,9060,1234FF")) { System.out.print(" [ " + s + " ] "); } } private static String[] split(String delim, String line) { StringTokenizer tokens = new StringTokenizer(line, delim); String[] v = new String[tokens.countTokens()]; int i = 0; while (tokens.hasMoreTokens()) { v[i] = tokens.nextToken(); i++; } return v; }

This doesn't work because my array will contain the delimiter also. It would look like this: [ RM^RES ] [ , ] [ 0013A2004081937F ] [ , ] [ , ] etc. Also, you changed my string to add a space between comma's, I can't alter the input as it's a protocol that must be followed.

bsd · Accepted Answer · 2013-07-11 09:15:56Z

I think you should not assume anything about the underlying delimiter.

 public static String[] split(String line, String delim) { Vector v = new Vector(); final String EMPTY_STRING = ""; StringTokenizer st = new StringTokenizer(line, delim, true); while (st.hasMoreTokens()) { String token = st.nextToken(); if (token.equals(delim)) { if (v.isEmpty() || v.size() > 0 && !EMPTY_STRING.equals(v.get(v.size() - 1))) { v.add(EMPTY_STRING); } } else { v.add(token); } } return (String[])v.toArray(new String[v.size()]); }

This doesn't work because every time I get a delimiter it will add an empty string and my array will look like this: [ RM^RES ] [ ] [ 0013A2004081937F ] [ ] [ ] etc. Basically every comma would become an empty array slot.

Collectives™ on Stack Overflow

Simulating String.split using StringTokenizer

5 Answers 5

Comments

3 Comments

Comments

1 Comment

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

5 Answers 5

Comments

3 Comments

Comments

1 Comment

1 Comment

Related