The producer-consumer pattern splits work between threads that create items and threads that process them, joined by a shared queue. It keeps a fast, steady task (such as reading from a sensor) from being blocked by a slow one (such as running a model). The queue itself is a Queue ADT.
Parts
- producer: creates items and puts them in the queue, for example a thread that reads samples from a radio or frames from a camera
- consumer: takes items out and processes them, for example a thread that runs a neural network
- shared queue: holds items between them
- mutex: only one thread may change the queue at a time
- condition variable: lets the consumer sleep until there is work, instead of checking in a loop
Why Use It
- a hardware stream must be read continuously, otherwise its buffer overflows and data is lost
- if one loop both reads and processes, every slow processing step stops the reading
- with two threads, the producer keeps reading while the consumer works
Bounded Queue And Backpressure
- if the consumer is slower than the producer, an unbounded queue grows until memory runs out
- a bounded queue (for example at most 3 items) forces a choice when it is full: drop the newest item, drop the oldest, or make the producer wait
- for live sensing, dropping old data is usually right, because fresh data matters most
C++ Example
// producer
{
std::lock_guard<std::mutex> lock(mtx);
if (queue.size() < 3) {
queue.push_back(std::move(item));
cv.notify_one();
}
}
// consumer
Item item;
{
std::unique_lock<std::mutex> lock(mtx);
cv.wait(lock, [] { return !queue.empty() || shutdown; });
if (shutdown) return;
item = std::move(queue.front());
queue.pop_front();
}
process(item); // slow work, outside the lockPoints to note:
- hold the lock only to push or pop, never during the slow work
waitwith a predicate handles spurious wakeups, because it checks the condition again after every wakeunique_lockis needed forwait, because waiting must unlock and relock the mutexstd::movepasses a large buffer without copying it
Related Ideas
- double buffering (ping-pong): two buffers, one being filled while the other is written out, which is a producer-consumer queue of size 2
- in Python,
queue.Queueprovides the locking, andmultiprocessingavoids the global interpreter lock for CPU-heavy consumers