1

The goal: pick a random number from a list every time a chunk is generated

Python code example to make it more clear.

I have this list:

listbytes = [87, 88, 89, 90]

And this function which splits the data:

def chunks(lst, n): "Yield successive chunks from lst, where n is a list of possible sizes" i = 0 while i < len(lst): k = min(random.choice(n), len(lst) - i) yield lst[i:i + k] i += k

And I call it this way:

for chunk in chunks(d, listbytes): ......

Every chunk created has a random size from 87 to 90, so: 1 chunk may have the size of 87, the next one may have the size of 90, and so on..

I have a similar function in Go which splits the data:

func split(buf []byte, lim int) [][]byte { var chunk []byte chunks := make([][]byte, 0, len(buf)/lim+1) for len(buf) >= lim { chunk, buf = buf[:lim], buf[lim:] chunks = append(chunks, chunk) } if len(buf) > 0 { chunks = append(chunks, buf[:len(buf)]) } return chunks }

The difference between this and the one in python is that the data is chunked using a fixed number. Example:

for _, chunk := range split(buf[:n], 100) { .......

This will chunk the data with a fixed size of 100, so every chunk has a size of 100, while with python is a random size from 87 to 90.

The desired output is the same thing: the chunks should have a random size everytime.

How can I achieve the same goal but in go?

Improve this question
3
  • 1
    can you explain clearly what is your question, input and desired output? do you need to get random element from listbytes = [87, 88, 89, 90] ? Commented Jul 27, 2021 at 15:46
  • An equivalent way of what is shown in python, yes. Get a random element from a list of numbers, which will be used to generate chunks. Everytime the chunks is generated should have a different size based on the picked element Commented Jul 27, 2021 at 15:54
  • use rand.Intn(n) with n equal to len(listbytes)-1 to generate a random index (let say i) and listbytes[i] to pick that from slice. Beware though that this sequence is deterministic, if you plan to use this to generate tokens or a simillar usecase, stackoverflow.com/questions/32349807/… should help you with that. Commented Jul 27, 2021 at 16:01

2 Answers 2

Reset to default
2

I think this is the Go version of your Python program (with a little main to run it):

package main import ( "fmt" "math/rand" "time" ) // split breaks buf into a slice of slices whose lengths // are randomly chosen from sizes, except for the last slice. // The last slice's length is less than or equal to a size that // was chosen from sizes, but may be larger than some other size // found in sizes. func split(r *rand.Rand, buf []byte, sizes []int) [][]byte { var chunk []byte chunks := make([][]byte, 0) for len(buf) > 0 { sz := sizes[r.Intn(len(sizes))] if sz > len(buf) { sz = len(buf) } chunk, buf = buf[:sz], buf[sz:] chunks = append(chunks, chunk) } return chunks } func main() { // Go's RNG is deterministic unless seeded with current time. s := int64(time.Now().Nanosecond()) r := rand.New(rand.NewSource(s)) sizes := []int{2,3,5,7} buf := []byte{0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25} fmt.Printf("result=%v\n", split(r, buf, sizes)) }

Here's a Go playground link for the same (so you can easily see and run): https://play.golang.org/p/U1vkPAKOQmI (Note that the time is always the same at the Go playground, so that it always runs the same).

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

2

A solution on top of io.Reader

main_test.go

package main import ( "bytes" "fmt" "io" "math/rand" "strings" "time" ) type rndReader struct { R io.Reader Rnd interface{ Intn(n int) int } buf []byte lastErr error } func (r *rndReader) Read(p []byte) (n int, err error) { if r.Rnd == nil { r.Rnd = rand.New(rand.NewSource(time.Now().Unix())) } if r.lastErr != nil && len(r.buf) < 1 { return 0, r.lastErr } // rather than a function argument (n), take len(p) j := len(p) if r.lastErr == nil { n, err = r.R.Read(p) // read to p r.buf = append(r.buf, p[:n]...) // save the read r.lastErr = err } p = p[:0] // reset p, prepare it to receive the random chunk k := min(r.Rnd.Intn(j), len(r.buf)) // select a random k p = append(p, r.buf[:k]...) // copy the desired portion from the internal buf to p copy(r.buf, r.buf[k:]) // re arrange the internal buffer r.buf = r.buf[:len(r.buf)-k] // adjust its len. return len(p), nil } func min(n, k int) int { if n > k { return k } return n } func ExampleFromStrings() { in := strings.Repeat(`12345`, 4) s := strings.NewReader(in) r := rndReader{R: s, Rnd: rand.New(rand.NewSource(1))} out := make([]byte, 3) for { n, err := r.Read(out) if err != nil { break } fmt.Printf(`n=%v err=%v buf="%s"`+"\n", n, err, out[:n]) } // Output: // n=2 err=<nil> buf="12" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="34" // n=2 err=<nil> buf="51" // n=1 err=<nil> buf="2" // n=0 err=<nil> buf="" // n=1 err=<nil> buf="3" // n=2 err=<nil> buf="45" // n=1 err=<nil> buf="1" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="23" // n=1 err=<nil> buf="4" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="51" // n=1 err=<nil> buf="2" // n=2 err=<nil> buf="34" // n=0 err=<nil> buf="" // n=1 err=<nil> buf="5" } func ExampleFromBytes() { in := []byte(strings.Repeat(`12345`, 4)) s := bytes.NewBuffer(in) r := rndReader{R: s, Rnd: rand.New(rand.NewSource(1))} out := make([]byte, 3) for { n, err := r.Read(out) if err != nil { break } fmt.Printf(`n=%v err=%v buf="%s"`+"\n", n, err, out[:n]) } // Output: // n=2 err=<nil> buf="12" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="34" // n=2 err=<nil> buf="51" // n=1 err=<nil> buf="2" // n=0 err=<nil> buf="" // n=1 err=<nil> buf="3" // n=2 err=<nil> buf="45" // n=1 err=<nil> buf="1" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="23" // n=1 err=<nil> buf="4" // n=0 err=<nil> buf="" // n=2 err=<nil> buf="51" // n=1 err=<nil> buf="2" // n=2 err=<nil> buf="34" // n=0 err=<nil> buf="" // n=1 err=<nil> buf="5" } func main() {}
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.