Browsing the blog archives for the year 2011.

Unsoliceted Code Review – Haskell Chat Server Edition

work safe

Our most recent Sep book club has been The Passionate Programmer. One of the things mentioned was taking some third party code and reviewing it.

This is not a new idea. Many people has said the best way to understand the code you write is to understand the good and bad of code written by other people. In addition, reading code in a language you are learning exposes you to functions you might not understand as well as idioms you don’t understand. We had an idea of meeting occasionally over lunch and tackling some code. Here’s kinda what I’m thinking that would look like. Please comment and let me know if there’s something else I’m missing.

With that in mind, I found an article on the Haskell Wiki where they implement a chat server in Haskell.

I’m only posting the code review for the final version. Technically, the final version and my changes to make it compile. There are more about that in the comments. I added my initials, BJB, to the comments I made and if I had to look up an API, I provided a link to where I learned about it.

This chat server has a couple of interesting features. It uses lightweight threading to handle multiple incoming and outgoing IO. It uses channels to communicate across these threads. The forking calls are also used to create separate “loops of control” for each IO channel being handled.

Below are the comments I’ve added to the program.

-- Socket based network library
-- http://www.haskell.org/ghc/docs/6.10.4/html/libraries/network/Network-Socket.html
import Network.Socket
-- System io calls.  Posix based
-- http://lambda.haskell.org/hp-tmp/docs/2011.2.0.0/ghc-doc/libraries/haskell2010-1.0.0.0/System-IO.html
import System.IO
-- for exceptions
-- http://haskell.org/ghc/docs/latest/html/libraries/base/Control-Exception.html
import Control.Exception
-- concurrent primitives
-- http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent.html
import Control.Concurrent
-- concurrent channels
-- http://www.haskell.org/ghc/docs/latest/html/libraries/base/Control-Concurrent-Chan.html
import Control.Concurrent.Chan
-- http://www.haskell.org/ghc/docs/latest/html/libararies/base/Control-Monad.html
import Control.Monad
-- http://www.haskell.org/ghc/docs/latest/html/libraries/base-4.4.0.0/Control-Monad-Fix.html
import Control.Monad.Fix (fix)

-- BJB - our message type will be an id and a string message to be passed to all
--     - channels not on that Id
type Msg = (Int, String)

-- BJB - The main function is run as an IO Monad
main :: IO ()
main = do
    -- BJB Create a new channel for the server side communication
    chan <- newChan
    -- create socket
    -- BJB - The socket is of type AF_INET http://en.wikipedia.org/wiki/AF_INET
    --       The socket type is Stream, which means they are connected, ie failure on
    --       on party breaks both connections
    sock <- socket AF_INET Stream 0
    -- make socket immediately reusable - eases debuggin
    -- BJB - Socket option SO_REUSEADDR
    --       http://publib.boulder.ibm.com/infocenter/iseries/v5r3/index.jsp?topic=%2Fapis%2Fssocko.htm
    setSocketOption sock ReuseAddr 1
    -- listen on TCP port 4242
    -- BJB - could be written as `bindSocket sock $ SockAddrInet 4242 iNADDR_ANY`
    --       iNADDR_ANY is the ipv4 wildcard.  4242 is the port
    --       this takes the socket and binds it to the network interface
    bindSocket sock (SockAddrInet 4242 iNADDR_ANY)
    -- listen on TCP port 4242
    -- BJB - could be written as `bindSocket sock $ SockAddrInet 4242 iNADDR_ANY`
    --       iNADDR_ANY is the ipv4 wildcard.  4242 is the port
    --       this takes the socket and binds it to the network
    listen sock 2
    -- BJB - recursive sink to pull data off the first chanel
    --     - this is to prevent the channel from filling it's
    --     - buffer without removing any data
    --     - See below for more information about fix and how it works
    forkIO $ fix $ \loop -> do
        (_, msg) <- readChan chan
        loop
    -- BJB - Passing the channel to the loop with an id
    mainLoop sock chan 0

-- handles all incoming connections
-- since it performs IO, it too must operate in the IO monad
mainLoop :: Socket -> Chan Msg -> Int -> IO ()
mainLoop sock chan nr = do
    -- accept on connection and handle it
    -- BJB - http://en.wikipedia.org/wiki/Berkeley_sockets#accept.28.29
    --       waits for a connection to a client
    --       a connection is a (Socket, SockAddr)
    conn <- accept sock
    -- BJB - passes the connection to the handler function
    --     - type of forkIO is IO() -> IO ThreadId
    --     - it's a lightweight thread
    --     - not for use if there are system threads excpected by the
    --     - underlying libraries
    --     - pass the channel to the socket handlers
    forkIO (runConn conn chan nr)
    -- BJB - loops back to accept the next connection
    --     - increment the connection id
    --     - the $! operator is defined as
    --     - f $! x = x `seq` f x
    --     - seq forces evaluation of a function, this increments
    --     - the connection id explicitly before the recursive call is made
    mainLoop sock chan $! nr+1

-- BJB Need to constrain the error handler used below to only receive an IOException
-- Will try to find out why later
errorHandler :: IOException -> IO()
-- BJB - the return () or return unit/void means do nothing
errorHandler _ = return ()

-- sends a message to the incomming socket
runConn :: (Socket, SockAddr) -> Chan Msg -> Int -> IO ()
runConn (sock, _) chan nr = do
    -- BJB - define a helper to broadcast a mesage to the channel
    --     - note: it closes over the channel
    let broadcast msg = writeChan chan (nr, msg)
    -- BJB - socketToHandle converts a network socket to a handle
    --     - a read / write handle
    hdl <- socketToHandle sock ReadWriteMode
    -- BJB - set to nobuffer, no need to flush
    hSetBuffering hdl NoBuffering
    -- BJB - As the user for their name
    hPutStrLn hdl "Hi, what's your name?"
    -- BJB - Store the user name
    name <- liftM init (hGetLine hdl)
    -- BJB - Tell everyone the user entered.
    --     - the ++ operator concats two lists efficientlyd
    broadcast ("--> " ++ name ++ " entered.")
    -- BJB - welcome the user on their socket
    hPutStrLn hdl ("Welcome, " ++ name ++ "!")
    -- BJB - duplicate the chanel
    --     - chan' is used to read
    --     - chan is used to write
    chan' <- dupChan chan
    -- fork off thread for reading fro the duplicate channel
    -- BJB - fix turns a lamba (taking a function as an arg and calling that at the end)
    --     -  into a loop
    --     - fix f = f (fix f)
    --     - http://en.wikibooks.org/wiki/Haskell/Fix_and_recursion
    --     - remember forkIO's threadId so we can work on it later
    reader <- forkIO $ fix $ \loop -> do
        (nr', line) <- readChan chan'
        -- BJB - if the message comes from a channel that isn't mine,
        --     - send it over the socket, otherwise skip it
        when (nr /= nr') $ hPutStrLn hdl line
        loop
    -- BJB - right now, this won't even compile
    --     - I suspect it isn't compiling because it's looking for an exception,
    --     - but one isn't present
    --     - handle is of type handle::Exception e => (e -> IO a) -> IO a -> IO a
    --     - the (e -> IO a) is an exception handler
    --     - However, the compiler wants it to be tightented down to only (IOException -> IO a)
    --     - this is done by using the errorHandler defined above as opposed to the
    --     - lambda from the original article
    handle errorHandler $fix $ \loop -> do
        -- BJB - http://en.wikibooks.org/wiki/Haskell/Monad_transformers#liftM
        --     - liftM turns init into a monad to take the line from the socket
        --     - init is used to kill the \n character
        line <- liftM init (hGetLine hdl)
        case line of
         -- BJB - quit if needed
         "quit" -> hPutStrLn hdl "Bye!"
         -- BJB - otherwise broadcast
         _      -> do
                   broadcast (name ++ ": " ++ line)
                   loop
    -- BJB - close the reader thread.
    --     - this isn't a problem because the previous line will not get here
    --     - until the writer chanel is closed from the 'quit' command
    killThread reader
    -- BJB - Let the room know this user left
    broadcast ("<-- " ++ name ++ " left.")
    -- BJB - close the socket/handle
    hClose hdl

Closing the circle.

work safe

I said I’d post a quick blurb on how things went in our experiment with an OpenSpace brown bag format.

In a word; well.

Here’s how we did it. I reserved three conference rooms with help from Kelly. Once we got together, we put some ideas on the board. We then did a quick show of hands to see what we wanted to discuss. Since we had a small group that wanted to talk about all of our three ideas, we just jumped right in and kept going until we finished. If we were larger or had a clearer split in desired topics, we would have split in to different rooms.

The had three main topics. Dasher, an internal pet project to get more visibility in to various project metrics around SEP, wanted to get some feedback. We discussed global variables in OO languages and their trade-offs. Our last topic was a discussion about practicing, learning not production use, Test Driven Development.

Would we do it again? Next time there is a hole in the brown bag schedule, I would.

Experimenting with Open Space Technology and ‘very little time’

work safe

I was wandering around the SEP.COMmons after passing a certification test on Friday and I noticed something. We had a whole in our brown-bag schedule. This will not stand! I wanted to try something out. I’ve seen Open Spaces work for conferences, but I was wondering how well they would work for a shorter time frame. Like an hour. With only two rooms.

My plan is to quickly explain the agenda, quickly get some ideas, quickly poll, and quickly break in to two groups to discuss those ideas.

I owe you one blog post about how well it went.

Read about Open Spaces on Wikipedia.

It has to be sexy and have legs

work safe

The point

I just got done with a company startup weekend. We were all coworkers and are all employee owners with no outsiders invited to participate. It was a blast and boy are my arms tiered!

This Friday afternoon before the pitch session, a group of us were talking about our project pitches and I mentioned that I felt the perfect startup weekend project is two things; It’s sexy and it has legs.

By sexy, I mean it appeals to my technical side. I want to play with new and exciting technologies. I want a toy that I’m proud to show off. I want geek-cred!

By having legs, I want to participate in a project I can see our company taking to market. I want a project I can see myself working on for a long time. Not because it is fun, but because I can see us making serious cheddar. Or at least enough to justify my salary.

Let’s take a look at one of the ideas I pitched (it failed to get picked up for the weekend by the way), an email client. It is not sexy, it’s email. I was trying to strip down outlook to its essentials. It doesn’t have legs. There isn’t an easy route to customers. It’s hard to be an indy dev. Especially when dealing with interfacing with exchange servers where your company already pays for Outlook. Besides, who gets excited over email? In my defense, I can rattle off Gmail, Sparrow, Eudora, and Mail.app that people love and the ones they hate like Outlook (express and office). It was gratifying, however, to notice how often people noticed they did hate Outlook over the weekend and say, “I should have voted for Brian’s project.” Though, I know they were just kidding.

The rest of the story

That said, I had a blast. I worked on an interesting project with a wonderful team. I didn’t get to play with the core of the project, but I did get to play with HTML5 and javascript. I didn’t know any of the stuff I did before I started, it just sounded like fun.

What did I learn playing with HTML5, JQuery, and Rx for Javascript? Browsers are way cooler than when I last did web development. DOM manipulation was crash prone in the early ’naughties. And pretty! Oh, wow. I did some canvas drawing and some file dropping. I finally got around to doing some JQuery. While I’d played with Rx for .NET, I used some of the techniques to handle dragging ant-trails on a canvas using the JavaScript release of the lib. And I realized why people write frameworks like Yesod and WebSharper. Even as an experienced dynamic languages dev, I had to track down more than a few type errors coming from unfamiliarity with the libraries.

More important than the stuff I worked on are the people I worked with. Working with a team is far different than working along side teammates. It may sound like I’m splitting hairs, but that statement resonates. We were holed up in a conference room with most of us working face to face across a table. This type of closeness does bother some, but it speed up the sharing of information. We changed the environment in which we work and it deeply effected how we work.

It was a great experience and I’m glad I did it. It was exhausting. I’ve heard someone mention we should do this every quarter, but I noticed he wasn’t participating. I like the idea of once a year. Otherwise, it’s too much like work.

Stoics vs Pollyanna-ists

work safe

Stoics believe in pure reason. One of the things that clouds their reason, they believe, is negative emotions. In attempting to remove themselves from the clutches of these negative emotions, they steep themselves in them to remove their power. It’s a type of desensitization therapy to regard a flower each time you see one in bloom as a thing that will wither and die. Just like you and everyone you love. In some ways, they are obsessed with those emotions.

There are another class of people that are kind of the opposite of the Stoics. They don’t steep themselves in negative emotions, they discount them. If I feel sad, it’s because I’m a weak person. It’s not quite the same as Pollyanna seeing the bright side, but they think it is.

I’ve noticed this recently with some personal emotional stuff and some people’s responses, my own included.

In addition, I’ve seen people behave like this on projects.

One school of thought is, “Fail early, Fail often.” The thought being we learn by making mistakes. They drive towards failure and focus their energy on understanding their failures more than their successes.

The other school of thought believes, “Failure is not an option.” Signs of failure are ignored as something not worth considering or they throw good money after bad thinking a failure can be overcome long after the failure has consumed all.

The truth is somewhere in the middle. We can learn from our successes and our failures. And most importantly, we must neither ignore our negative emotions nor stew in them to the exclusion of enjoying the beauty around us.

Try, Try Again – What a Little Code Can Mean

work safe

I have started my Erlang based MUD a few times. The first time, I got bogged down in telnet details. The second time, I got lost building far too many proxy objects and indirections. Then I wasn’t even sure what I’d written was correct or that my changes wouldn’t break something.

I was having a hard time figuring out how to drive out the code. Where do I start?

This time around, I’ve gotten farther than I ever had before. I’m at a stage were I can move my character from one room to another. The perspective this time, I got a chance to drive the code out by running it via the REPL. I found a happy medium between top down and bottom up. This is similar to TDD use to test out an View-Model. My REPL based code had simple functions that mimicked the future texted based input. Seems like a duh now that I’ve said it. It really does feel more like I’m cutting with the grain.

But now, how do I make sure that I’ve got code that works constantly? Especially when I start dealing with more complex behaviors. I want to “record” my REPL sessions while focusing on building contexts since there are so many ways to build up a player and rooms. To do that, I’ve built a BDD micro framework based on E-Unit and a dash of rspec. I call it “test.hrl” and it is all of three macros!

-include_lib("eunit/include/eunit.hrl").
-define(It(Text,Func), {"It " ++ Text, Func}).
-define(It(Text,Setup,Cleanup,Func),
        {"It " ++ Text, setup,Setup,Cleanup,Func}).
-define(Describe(Text,Tests),{"Describe " ++ Text, Tests}).

It is based on the fact e-unit can used nested test descriptions to run tests. I just wrote some macros around it to make it fit my preconceptions of what a bdd framework should be like using terms I like.

When used, it looks like this

-module(player_tests).
-include("tests.hrl").

player_test_() ->[
  ?Describe("Bad Password",
    [?It("should return an error",fun setup/0,fun cleanup/1,
          ?_test(begin ?assertEqual(error, player:login("Tony", "BassPassword"))end))
    ]),
  ?Describe("Good Password",
    [?It("should have a player proxy",fun setup/0,fun cleanup/1,
                    ?_test(begin Me = player:login("Tony", "Hello"),
                                 ?assertEqual({ok,"You aint got jack!"},
                                 Me:inventory())
                           end))
     ]),
  ?Describe("Room Interaction",
    [?It("should describe the lobby",fun setup/0, fun cleanup/1,
         ?_test(begin Me = player:login("Tony", "Hello"),
                      ?assertEqual({ok, "It's a lobby"},
                                   Me:look())
                end)),
     ?It("should move to the kitchen", fun setup/0, fun cleanup/1,
         ?_test(begin Me = player:login("Tony", "Hello"),
                      Me:move("north"),
                      ?assertEqual({ok, "It's a kitchen"},
                                   Me:look()) end))])
].

setup() ->
        % I don't know why, but I need the print
        % to make the kitchen test pass
        io:format(""),
        stubs:fake_rooms().
cleanup(_Pid) ->
        stubs:stop_fake_rooms(),
        true.

Here’s my test runner

-module(test_runner).
-export([run/0]).
-include_lib("eunit/include/eunit.hrl").

run() ->
        eunit:test([player_tests],[verbose]).

The output looks like this

erl -noshell -pa ebin -s mnesia start -s test_runner run -s init stop
======================== EUnit ========================
module 'player_tests'
  Describe Bad Password
    It should return an error
      player_tests:7: player_test_...ok
      [done in 0.016 s]
    [done in 0.016 s]
  Describe Good Password
    It should have a player proxy
      player_tests:11: player_test_...ok
      [done in 0.015 s]
    [done in 0.015 s]
  Describe Room Interaction
    It should describe the lobby
      player_tests:18: player_test_...ok
      [done in 0.016 s]
    It should move to the kitchen
      player_tests:23: player_test_...ok
      [done in 0.016 s]
    [done in 0.032 s]
  [done in 0.063 s]
=======================================================
  All 4 tests passed.

This may not be perfect. I may not be the most ergonomic. I know it isn’t. But it’s close enough for me right now. I feel like I can describe what I want in a manner that fits me. Three macros and a slight change in how I view the world means I finally have this project moving forward well. It is truly amazing what just a little code can do.

My goal, make you functional in 12 hours

work safe

I’m still thinking about teaching F# to my fellow coworkers later this year. In order to do that, I’ve had to start thinking about what to teach.

This class is a tough assignment for me. The people who want to take it have said, “No parsing!” evidently, parsing is for eggheads and dorks (Fuller is such a reactionary anti-academic). And I have to make it useful! Fine. My thesis is this; F# is best when used for libraries, when used for asynchronous programming and reactive manipulation of streams. Also, functional abstractions and immutability are great for data-centric programming.

Below is my rough syllabus. Please feel free to comment if you think something is missing. It almost surely is.

  • Week one - Making sure you have F#, Basic Syntax, type inference, recursion. There will be a reading assignment to accompany this. In addition, I’ll have a github project up that will have some tests that need to pass. I’m calling these “worksheet” problems. Week two - Pattern matching, higher order functions, maps and folds. A few worksheet problems added to github. A game of life mini-project assignment? Week three - Algebreaic types, records. In addition, we will start dealing with tail-calls and branching recursion. Worksheets. Week four - Streams and Sequences. These are kind of what functional programming is about, to my mind at least. It will allow spring boarding to reactive (eventually). They will write a few streams in worksheets. Week five - Modules, Objects, and Mutable refs. We’ll write a lib that will be used by a CLR program to do the “Flocking Birds” Week six - Asynchronous Workflows. We’ll write a command line podcatcher that will be totally asynchronous, list of urls in a file spits out to RSS feeds, to saved mp3 files. Week seven - Mailboxes. A simple Spreadsheet (parser provided) will allow gets to happen threadsafe from the updates. A worksheet with some massive parallel updates of a queue where everything must be handled thread safe. Week eight & nine - Custom Workflows. You know… Monads. May take a couple weeks. Still no damn idea what I’m doing here, honestly. I need some workflow examples to implement. Week ten - Reactive Programming. We’ll write a generative seqencer that uses a WPF control and a sound lib I’ll provide. Week eleven - back to C#. A view of linq and lambdas. Week twelve - Topics in F#. Lightning talks! And a fluid simulation project. IN 3D! Again, I’ll provide the control.

Oh, theres so much to be fleshed out, but here is where I’m thinking of starting.

And I need to do the projects to make sure they are sized right.

Picking a functional language

work safe

Somehow, I’ve been suckered and bamboozled. I’ve been talked in to prepping an internal course on Functional Programming. To talk about functional programming, I need to have the students actually program functionally. There are several choices I’m considering; FSharp, Clojure, Haskell, and Erlang. There are plenty of others, but I’m not as familiar with them so they won’t be considered. They each have their own strengths and weakness.

Clojure was started on top of the JVM (Java Virtual Machine), but is also being ported to the CLR (Common Language Runtime). It’s a dynamic functional language that is a part of the Lisp family of languages. There’s a lot to be said for it. It’s got Macros. It’s got all kinds of documentation. It’s got all kinds of libraries. It has great IDE support. It even has a bunch of Emacs support as well.

Haskell is a research language that’s crossed in to industry. It compiles to machine code. It is strongly typed, way stronger typed than Csharp or Java. It is type inferred. It is Lazy and Pure. There is a community of developers releasing libraries. There are some really good documents as well. Many interesting functional research papers use Haskell as their implementation language.

Erlang is an industry secret that escaped to main stream use. It is a dynamic language that runs on it’s own interpreter. It has a bit parsing syntax. It has strong networking support. There is a growing community and some good documentation. It has a phenomenal set of libraries for concurrent programming.

FSharp is a fairly new language from Microsoft. It’s a strongly typed and type inferred language. It has some great language extension features like workflows and quoted code. It is a variant of Ocaml. It lives on the CLR and integrates well as a consumer of .NET libraries or as a .NET library to be consumed by C# code.

They are all very different and have different focuses, so I had to eliminate some. Erlang is out because it is almost too focused on concurrency to the point of distraction. Haskell is a wonderful language but has a sovereign ecosystem that would preclude it from being used with the majority of our work. Clojure is wonderful and I can turn a bunch of existing material from Scheme or Lisp without much work, but it is mainly a JVM language and I can’t vouch for the completeness of the CLR implementation at this time.

This leaves FSharp as my language of choice. It plays well with .NET and we have a LOT of .NET projects. I can use a lot of Haskell resources as a starting point. It should work well. I’m looking forward to class.