Freelance Web Developer

Jul 3, 2015

Job queue example in Golang

What is a Job Queue?
A job queue, is a data structure maintained by job scheduler software containing jobs to run. Users submit their programs that they want executed, "jobs", to the queue for batch processing. The scheduler software maintains the queue as the pool of jobs available for it to run.
via Wikipedia

You can write highly concurrent Job queue system using Golang.
Here is the example code by @blackecaldwell


// Example of a simple job queue
// - Each job is a series of tasks.
// - We finish all of a job's tasks before moving onto the next one.
// Blake Caldwell

package main

import (

// Run a job - blocks until the job is done
func runJob(jobQueue chan chan string, name string) {

 // our job is a series of tasks
 taskQueue := make(chan string)
 defer close(taskQueue)

 // Queue our tasks in the job queue
 jobQueue <- taskQueue

 // job queue made room for us - queue up each of our tasks
 fmt.Printf("Starting job %s\n", name)
 for i := 1; i <= 5; i++ {
  taskQueue <- fmt.Sprintf("%s: %d", name, i)
  time.Sleep(1000 * time.Millisecond)
 fmt.Printf("Job %s done!\n\n", name)


func main() {

 // This is our job queue - a channel of channels. We write channels of tasks into it.
 jobQueue := make(chan chan string)
 defer close(jobQueue)

 // simulate multiple requests to use the job queue
 fmt.Println("Begin: Kicking off workers that want to use the job queue")
 go runJob(jobQueue, "A")
 go runJob(jobQueue, "B")
 go runJob(jobQueue, "C")
 fmt.Println("End: Kicking off workers that want to use the job queue\n")

 // this is just for the example - this would be handled differently in a real service
 remainingJobs := 3

 // handle each job in the queue
 for job := range jobQueue {

  // Execute each of the tasks
  for task := range job {
   fmt.Printf("Handling: %s\n", task)

  // break out when done
  if remainingJobs == 0 {


Try this example Update: Here is an more detailed code by Nick Saika

Labels: ,

By :