Let’s dumb down a code example from Golang.org

I was very confused trying to learn the computer programming language called Go. I was following the “Codewalk” example from golang.org named “Communicate by Sharing Memory” example, which is here: https://golang.org/doc/codewalk/sharemem/

So I rewrote it to be dumb. Like old C or something. So I could understand it.


// Copyright 2010 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// modified by don bright 2017

package main

import (
	"fmt"
	"net/http"
	"time"
)

type Url string

type HTTPState struct {
	url    Url
	status string
}

// HTTPStateMonitor maintains a map that stores the HTTPState of 
// the URLs being polled, and prints the current HTTPState every 
// few seconds. It takes a chan HTTPState to which Url HTTPState 
// should be sent.
func HTTPStateMonitor(stateupdates chan *HTTPState, done chan int) {
	urlstatus := make(map[Url]string)
	ticker := time.NewTicker(time.Second * 5)
	count := 0
	for {
		select {
		case  3 {
				done <- 1
			}
		case s := <-stateupdates:
			urlstatus[s.url] = s.status
		}
	}
}

func Poller(in chan Url, out chan Url, stupdates chan<- *HTTPState){
	for url := range in {
		fmt.Println("poller", string(url))
		resp, err := http.Head(string(url))
		s := resp.Status
		if err != nil {
			s = err.Error()
		}
		stupdates <- &HTTPState{url, s}
		out <- url
	}
}

func Recycler(in <-chan Url, out chan<- Url) {
	for url := range in {
		fmt.Println("recycler", string(url))
		go func(urly Url, outy chan<- Url) {
			time.Sleep(time.Second * 5)
			outy <- urly
		}(url, out)
	}
}

func main() {
	stateupdates := make(chan *HTTPState)
	pending := make(chan Url)
	complete := make(chan Url)
	done := make(chan int)
	go func() {
		pending <- "http://www.google.com"
		pending <- "http://golang.org"
		pending <- "http://blog.golang.org"
	}()
	go HTTPStateMonitor(stateupdates, done)
	go Poller(pending, complete, stateupdates)
	go Recycler(complete, pending)
	_ = <-done
}

 

OK.

So the first thing, is that all the channels are set up before the program runs.

Then the channels are initialized with some data, in this case three internet website address urls.

Then I thought of the program as having three “Processors” running at the same time. Not CPU processors, not processes in an Operating System, just generic “Processors”, like people who perform some function as part of a group. Each processor has it’s own sort of special thing it does, and it communicates with the others.

What does it actually do? What is the actual output? Well, it gets the ‘HTTP status’ of each website, and prints it out, over and over.

 

 don@serebryanya:/tmp$ go run y.go
 poller http://www.google.com
 poller http://golang.org
 recycler http://www.google.com
 poller http://blog.golang.org
 recycler http://golang.org
 recycler http://blog.golang.org
 http://www.google.com 200 OK
 http://golang.org 200 OK
 http://blog.golang.org 200 OK

 

The cool part is that instead of “going through a list” data structure to print out the urls, it instead “reads items from a channel”. So the Channel itself kind of replaces the idea of the centralized data structure.

Here is a nice Stack Overflow article about this kind of phenomenon.

http://stackoverflow.com/questions/10424144/is-it-possible-to-use-gos-buffered-channel-as-a-thread-safe-queue

Basically, a channel can kind of replace a queue. The trick appears to be in changing how I think about the data in my program. Let go of the idea of having a central queue that various processors can read and write data to. That would use the old ‘mutex’ idea.

Instead, try to think about passing data between processors by using channels. I can kind of feel my head starting to wrap around that idea a little bit. It’s still confusing though.

In the example above… it is really confusing because the way they set up channels is so that you basically feed one channel into another, then back into itself, in a way such that you never have to ‘close’ the channels. Fascinating and weird.

The big part of my ‘rewrite’ is to give a name to what they are doing when they set up this fascinating feed back loop. I called it ‘Recycling’. So I have basically Three Processors.

Processor One, Monitor: read data (from a channel) regarding the States of each website, and print a report

Processor Two, Poller: fetch HTTP headers from each website, and feed the data to Processor One using a channel. But… “Each website” is given by a url read from another channel, the input channel. Once fetching is complete, it writes the url onto another channel, the output channel.

Processor Three: This basically does the same thing as Processor Two, it reads urls from a channel, does something, and then writes them out on another channel. Instead of fetching data, though, it just sleeps.

The trick here is that in Main(), when setting all this up, we feed the output channel from Processor Two as the input channel to Processor Three, and vice versa, like some kind of Ourobouros (snakes eating each other’s tails).

z

In this picture… it is the URLs themselves, simple golang ‘string’ types,  that travel around. They are sometimes going through the ‘fetch’ process, and sometimes going through the ‘sleep’ process. Each process reads the url from an incoming channel, processes it, and the puts the url onto an output channel.

The two ‘mouths’ are where the urls transfer between processors… these represent the moment when the ‘fetch’ process is reading its input channel at the same moment the ‘sleep’ processor is writing to it’s output channel. And vice versa. Do you see?

The other trick here is that reading and writing on channels is a “blocking” activity – it freezes the processor that is doing it. The processor just sits there waiting. So you can see that you will need two processors to get anything done here… one must read while the other writes, and vice versa. Otherwise things will pause forever.

Now I simplified this alot.. the actual program uses goroutines inside the Processors… but that doesn’t change the basic idea too much, I hope. The golang.org authors have cleverly allowed this to happen in the way they combine goroutines and Sleep and “range” style reading of channels. I have dumbed it down a little so that it is easier for old C++ people like me to have a look at it. But I definitely felt like I was getting a slightly better grasp of what they were doing.

And that is that! Thanks for reading.

Posted in Uncategorized | Leave a comment