0

I find myself repeating this pattern and have often wondered if it is idiomatic in Java or there is a better way of achieving this behaviour.

Problem: Given a producer/consumer setup, the consumer wants to process batches of items, so it uses drainTo(), however drainTo() will poll for existing items and possibly fail to get any items, to avoid this I prefix the drain with a take() to ensure it blocks until at least one item is available.

One problem I get, with a particular dataset, is with many use cases that the batch size is often irregular alternating between (1, N, 1, N). In general is this a common way to solve this problem:

Example:

 ArrayBlockingQueue<Foo> queue; function void produce() { while(true) { queue.put(createFoo()); } } function void consumeBatchSpin() { while(true) { List<Foo> batch = Lists.newLinkedList(); queue.drainTo(batch); doSomething(batch); //the problem here is that if nothing is being produced, this loop will spin } } function void consumeBatchTake() { while(true) { List<Foo> batch = Lists.newLinkedList(); batch.add(queue.take()); //force at least one item to be there queue.drainTo(batch); doSomething(batch); } } 
1

1 Answer 1

1

Have you considered adding to a list and taking the whole list on get.

I have posted one here recently. It is undergoing code review here but my tests suggest it is robust.

Essentially, when you do a put you add your new element to the current list. When you do a get you get the whole list and atomically replace it with a new empty one.

No need to use drainTo and no spinning at all.

Sign up to request clarification or add additional context in comments.

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.