Concat remaining bytes from a byte[] to a new full byte[]

Question

I have a TCP stream coming in and it fills up a 256 byte[].

From there I process the byte array parsing out messages etc.

Once I get to < 100 bytes in the first array, I want to take the remaining bytes add them to a new array and then append the new 256 byte array and continue processing so I don't miss messages.

 public static byte[] Combine(byte[] first, byte[] second) { byte[] ret = new byte[first.Length + second.Length]; Buffer.BlockCopy(first, 0, ret, 0, first.Length); Buffer.BlockCopy(second, 0, ret, first.Length, second.Length); return ret; }

I'm using this function (Taken from one of Jon Skeet's post) however there is an issue that happens in that the byte[] continually gets bigger.

For example if I have buffer[] that is 256 and newbuff[] 256 and pass it to the function above...I get back a ret of 512[].

Now if I pass the Combine function again it adds on the 256 to the 512 and continuously grows causing a bit of an issue as I am processing a rather large tcp data feed.

Any suggestions on how to make this more efficient? Currently I have tried using this variation but it seems like I am caught in a circle.

 public static byte[] Combine(byte[] first, byte[] second, int srcOffset) { byte[] ret = new byte[first.Length -srcOffset + second.Length]; Buffer.BlockCopy(first, srcOffset, ret, 0, first.Length - srcOffset); Buffer.BlockCopy(second, 0, ret, first.Length - srcOffset, second.Length); return ret; }

Thanks in advance guys!

Your basic goal builds the memory inflation right into the algorithm. You state that your goal is to always take 256, subtract only 100, and then add another 256. Your initial algorithm doesn't appear to even subtract the 100 bytes, which naturally leads to your consistent increase of 256. The second will still only subtract that offset. Unless that offset is at least the size of the second byte array on average, your array is going to keep growing. For reference, consider this very similar problem from 1987. — Mark Bailey
– Mark Bailey, Commented Jun 7, 2016 at 21:12
@MarkBailey I'm not sure what you are saying because it seems to me that you're telling me the same thing I said which is I know the first function keeps growing in size. I also said that I didn't write that function. I made a modification to it on the second hoping to solve the issue of (Adding remaining bytes on to a new array) which is what it issue is. I'm tired and came back from a death anniversary. You may be completely right just not sure....seems like we're saying the same thing to me... — Valmorgal
– Valmorgal, Commented Jun 8, 2016 at 6:34
The second function provides more ease in paring down the prior byte array, but neither solves your problem entirely by themselves. In either case some logic must exist outside of the function to determine when to trim the byte array: in the first case by feeding in an already trimmed array; in the second case by choosing the size of srcOffset. Again, unless that offset is at least the size of the second byte array on average, your array is going to keep growing. — Mark Bailey
– Mark Bailey, Commented Jun 11, 2016 at 17:06

Peter4499 · Accepted Answer · 2016-06-07 20:46:06Z

1

Maybe this is over simplifying it you can do something like the below, I personally use memory streams withing TCP servers to keep all the data and it's worked out well.

 public static byte[] Combine(byte[] first, byte[] second, int srcOffset) { MemoryStream result = new MemoryStream(); result.Write(first, 0, first.Length); result.Write(second, 0, srcOffset); return result.ToArray(); }

answered Jun 7, 2016 at 20:46

Peter4499

6857 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Valmorgal Over a year ago

Thanks for replying....actually no it's not over simplifying it.....but how do you deal with 0,0,0,0's in the data stream ....that'd be awesome and probably solve all my worries Peter..:)

Peter4499 Over a year ago

@Valmorgal the reason for the 0's is (I'm guessing ) you're not checking how many bytes are read. When you're reading from the tcp stream you have an return of an integer. That's how many bytes were read. So even though you have a 512 byte buffer it may only read 30 bytes. That means that the last 482 bytes in the array will be 0's. The way i read your question is that the param first is a full array and second only has srcOffset number of bytes

Valmorgal Over a year ago

You're right...how would you suggest I go about checking how many bytes I read...because I think this is a part of the problem. If I am copying the right data it may end up being 0x54,0x83...etc...instead of 0x00,0x00,0x83,0x45.....0x00,0x00... I feel like I'm missing a part to this check?...

Collectives™ on Stack Overflow

Concat remaining bytes from a byte[] to a new full byte[]

1 Answer 1

3 Comments

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

3 Comments

Related