If I want to maintain a queue of frames of images on the server side that I will be sending to the client what data structure should I use ?
I am trying to make a simple app where I will send frames to the server and server will then push them to other clients.
should I maintain this queue as an atom or as a ref?
You could just use one of the queue classes from java.util.concurrent. Easy access to the Java standard library is after all one of the strong points of Clojure, so you don't have to build everything yourself from the building blocks provided by Clojure if Java already provides something that does the job.
I would suggest to choose something from the implementations of the interface BlockingQueue.
LinkedBlockingQueue to keep track of work which needs to be processed. As the agents process tasks, some of the tasks result in pushing more new tasks on the queue.The most important operation of a queue is popping items: namely getting-and-removing an item from the collection.
As this operation is composite, refs are more naturally suited to perform it atomically, this is - preventing race conditions (e.g. two threads getting the same item).
(defn remove-and-get [queue] (dosync (let [i (peek @queue)] (alter queue pop) i))) (def q (ref (clojure.lang.PersistentQueue/EMPTY))) (dosync (commute q conj 42) (commute q conj :foo) (commute q conj [])) [(seq @q) (remove-and-get q) (seq @q)] ;; evals to [(42 :foo []) 42 (:foo [])] Equivalent functionality can be implemented in terms of atoms as well.
(defn remove-and-get [queue] (let [snapshot @queue i (peek snapshot)] (if (compare-and-set! queue snapshot (pop snapshot)) i (recur queue)))) (def q (atom (clojure.lang.PersistentQueue/EMPTY))) (swap! q conj 42) (swap! q conj :foo) (swap! q conj []) [(seq @q) (remove-and-get q) (seq @q)] 
ref guarantees transactional consistency (think multiple refs in one dosync) but that will still allow multiple threads to read the same value at a given point in time. The second example will probably work but is somewhat suboptimal as it may run multiple times and it may not be very efficient to compare a (very long) list for equality on each iteration.Seems to be a possibility for core.async.
Even though Clojure gives you great data structures and primitives, it won't save you from coding race conditions. Check out these examples which won't work:
;; WILL NOT WORK (def queue (atom '(:foo :bar :baz :qux))) ;; the code below is called from threads ;; take the first value of the "queue", derefing it (let [peeked-val (first @queue)] (do-something-with peeked-val) ;; update the atom to remove the first value ;; DANGER: You've derefed the value above but you're swapping it in a separate step ;; (here). Other threads may have read the same value or may have mutated it ;; in the meantime! (swap! queue rest)) What about refs?
;; WILL NOT WORK (def queue (ref '(:foo :bar :baz :qux))) ;; the code below is called from threads ;; take the first value of the "queue", derefing it, this time in a transaction! (dosync (let [peeked-val (first @queue)] (do-something-with peeked-val) ;; update the ref to remove the first value in the transaction ;; DANGER: Refs give you transactional consistency (i.e. consistency ;; between read/write access to other refs) but this doesn't really apply ;; here as we only have on ref. Other threads may have read the same value ;; or may have mutated it in the meantime! (alter queue rest))) You can accomplish this with Clojure's data structures and atoms. The key is to use swap-vals! so you only touch the atom once - otherwise you'll run into race conditions as you have two operations as in the example above: Derefing the atom (getting its value) and swapping it (changing its value).
(def queue (atom '(:foo :bar :baz :qux))) ;; use `swap-vals!` on atoms to get both the old and new values of the atom. ;; perfect if you want to peek (get the first value) while doing a pop (removing ;; the first value from the atom, thereby mutating it) ;; use the code below in threads (or somewhere in core.async) ;; it's somewhat non-idiomatic to use `rest` and `first`, see the text below for ;; other options (let [[old new] (swap-vals! queue rest)] (println "here's the popped value:" (first old))) You could also use a PersistentQueue instead, construct it with (clojure.lang.PersistentQueue/EMPTY) - then you can call peek and pop on it instead of first and rest. Don't forget to put it in an atom just like the list above :)
You could also use something like a java.util.concurrent.LinkedBlockingDeque. Check out this commit which introduces parallel building capabilities to the ClojureScript compiler for an example of using a LinkedBlockingDeque.
You could try agents, the idea being the following:
For each client you have one agent, and you simply send-off the command to transmit the frame to the client. Since actions on agents are executed in FIFO-order (at least as long as you have just one sending thread).
(send-off client transmit-frame frame) 