I’ve been working with our corporate website for the past few days. I’m trying to get analytics for some A/B split testing. As a part of that, I got to play with some really neat FSharp features. Type providers have allowed me to use a snippet of json to build a type hierarchy without doing the heavy lifting myself. And recursive sequences allow me to consume a paginated webservice as though it was a single data set.

Bringing My Customers Together

The data I’m currently interested in is, as far as I can discover, only available by asking each Hubspot contact for their list of form submissions so I can count the number of submissions to each form. But I can only ask for 100 customers at a time. The response will tell me if I can fetch more and the vid-offset I can use to get to the right spot in the pagination.

let rec Contacts offset =
   seq { let data = HubspotContacts.Parse(Http.RequestString("http://api.hubapi.com/contacts/v1/lists/all/contacts/all",
                                                             // apikey not included ;)
                                                             query=["hapikey", apikey; "count", "100"; "vidOffset", offset],
                                                             headers=[Accept HttpContentTypes.Json]))
         yield! data.Contacts
         if data.HasMore then
            yield! Contacts (data.VidOffset.ToString())}

What’s happening? The seq structure creates a seq computation (think IEnumerable) and allows us to write code that generates a seq. The yield! keywork takes a collection and doles each element out in turn. If we find the collection has more, we yield the results of a call with the offset. This means our first call will use the offset “”.

Types are your friends

FSharp type providers are wonderful. They can, at compile time, parse out some code and generate a type tree. In particular, using the FSharp.Data library I can give it a snippet of JSON and get back a strongly typed parser.

// I'm leaving a lot out of the json snippet.
// You don't need to see the details of my customer data
let [] ExampleResponse = """ { "contacts" : [ {"form-submissions": [{ ... }]}
                                                       { ...} ],
                                        "has-more" : true,
                                        "vid-offset": 1234 } """
type HubspotContacts = JsonProvider

Reductio ad absurdum

The only thing left for me to do is take this and map / reduce my way to a collection of form-ids and counts.

Contacts ""
// Transform from a seq of contacts to a seq of form submissions
|> Seq.map (fun c -> c.FormSubmissions)
|> Seq.concat
// Extract the form id
|> Seq.map (fun fs -> fs.FormId)
// Collect the counts of unique formIds
|> Seq.fold (fun s v -> Map.tryFind v s with
                        | Some(count) -> Map.add v (count+1) s
                        | None -> Map.add v 1 s) Map.empty
// Report the FormId and Count
|> Map.iter (fun formId count -> printfn "%A %i" formId count)