The producer-consumer pattern splits work between threads that create items and threads that process them, joined by a shared queue. It keeps a fast, steady task (such as reading from a sensor) from being blocked by a slow one (such as running a model). The queue itself is a Queue ADT.

Parts

  • producer: creates items and puts them in the queue, for example a thread that reads samples from a radio or frames from a camera
  • consumer: takes items out and processes them, for example a thread that runs a neural network
  • shared queue: holds items between them
  • mutex: only one thread may change the queue at a time
  • condition variable: lets the consumer sleep until there is work, instead of checking in a loop

Why Use It

  • a hardware stream must be read continuously, otherwise its buffer overflows and data is lost
  • if one loop both reads and processes, every slow processing step stops the reading
  • with two threads, the producer keeps reading while the consumer works

Bounded Queue And Backpressure

  • if the consumer is slower than the producer, an unbounded queue grows until memory runs out
  • a bounded queue (for example at most 3 items) forces a choice when it is full: drop the newest item, drop the oldest, or make the producer wait
  • for live sensing, dropping old data is usually right, because fresh data matters most

C++ Example

// producer
{
    std::lock_guard<std::mutex> lock(mtx);
    if (queue.size() < 3) {
        queue.push_back(std::move(item));
        cv.notify_one();
    }
}
 
// consumer
Item item;
{
    std::unique_lock<std::mutex> lock(mtx);
    cv.wait(lock, [] { return !queue.empty() || shutdown; });
    if (shutdown) return;
    item = std::move(queue.front());
    queue.pop_front();
}
process(item);        // slow work, outside the lock

Points to note:

  • hold the lock only to push or pop, never during the slow work
  • wait with a predicate handles spurious wakeups, because it checks the condition again after every wake
  • unique_lock is needed for wait, because waiting must unlock and relock the mutex
  • std::move passes a large buffer without copying it
  • double buffering (ping-pong): two buffers, one being filled while the other is written out, which is a producer-consumer queue of size 2
  • in Python, queue.Queue provides the locking, and multiprocessing avoids the global interpreter lock for CPU-heavy consumers