F#: TypeProviders + Basic Operators

I have recently been learning a bit of F# on my spare time, working on a few hobby projects to do so. As a C# developer by day, I think it is wise to learn other technology, both to be more attractive as a consultant, but also to stay sharp and continue to learn new ways of thinking. F# and functional programming, in addition to being fun to work with, provides a lot of useful techniques and mindsets, which can be utilized when working with other languages.

In this post I would like to showcase a few nifty features in F#. I will use examples from a console app, which reads csv-files containing financial data, massages it, and outputs stats regarding total value, ROI and charting. F# provides a very neat feature which makes reading and parsing data sources very easy. I will also give an introduction to some of the basic features of F# that does not exist in languages like C#. As I have just started learning F# myself, this post is clearly not intended for functional programming gods, but for those unfamiliar with the language or the concept of FP.

Type Providers

The app reads csv-files containing monthly portfolio updates, parsing these and then computes development in regards to the previous month, and charts these numbers. This is achievable in quite few lines of code, mainly enabled by the built in csv type provider. This magically gives us strongly typed access to the csv files in a single line, requiring minimal setup.

 type StatusUpdate = CsvProvider<"template.csv">

The CsvProvider is used to create the type corresponding to the csv-files header columns. The type now has a member function Load taking the path of a file on the specified format as argument, this can be the same file as referenced in the type provider or another similar file. I have typically used one template file kept as part of the source code, then reference actual "production" files in a configurable location. After loading a file we get an object containing all the rows of the file, with intellisense on the column names.

Function composition

In my program I would like to load all files in a directory and get an array containing the rows of all the files. F# has a very useful feature, function composition (the >>-operator), which takes 2 functions and creates a pipeline resulting in a new function. This can be used to create a function taking in a file name, expanding the path, loading the file using the newly created csv file type and selecting the rows from each file.

 let parseTransactionFile = 
    (Path.GetFullPath >> StatusUpdate.Load >> (fun (s:StatusUpdate) -> s.Rows |> Seq.toList))

The parseTransactionFile-function is composed from 3 other functions. Path.GetFullPath takes a string and returns a new string, StatusUpdate.Load takes the string and loads a csv provider type which is passed to a lambda function selecting the rows of the csv file as a list.

Pattern matching

Finally I would like to be able to provide a file directory path, and enumerate all the files in the directory, calling the newly created parseTransactionFile-function which each filename as argument, returning a flattened list containing all the rows of the files in the directory. For this, F# has two useful operators, match ... with and |>. Match takes in an argument and matches it with a pattern and executes the defined expression.

 match <something> with 
| pattern1 -> expression1 
| pattern2 -> expression2 

This is also achievable in C# now, as pattern matching was introduced in C#7, basically supercharging switch statements! The when keyword in C# (previously known only from exception filters in C#6?) is pretty similar to the with keyword in F# See Pattern Matching in C# 7.0 Case Blocks.

switch (spaceItem)
{
  case Star s:
    AvoidHeatSource(s);
    break;
  case Planet p when (p.Type != PlanetType.GasGiant):
    LandSpacecraft(p);
    break;
  case null:
    break;
}

Pipe

The pipe forward |> or pipe backward <| operators are simply another way of applying functions to arguments. In F# functions are defined and invoked as follows

let square x = x * x
let squareOf5 = square 5

Multiple functions would normally be nested using parentheses giving us in example:

let square x = x * x
let times2 x = x * 2
let add2 x = x + 2
let result = add2 (times2 (square 5))

This is similar to to how functions are nested in other non functional languages, and as we add more functions to the pipeline, it gets messy. Therefore we can use the pipe operator instead, making the code much more readable:

let result = 5 |> square |> times2 |> add2

We use pattern matching and pipe to create our function reading all files from a directory:

 let loadAllStatusUpdates fileDirectory = 
    match Directory.GetFiles(fileDirectory, "*.csv") with
    | [||] -> [||]
    | files -> files |> Array.map parseTransactionFile
    |> List.concat

Directory.GetFiles(fileDirectory, "*.csv") returns a sequence of strings - the file names of all the csv-files in the provided directory path. This sequence is then used an argument in a pattern match, if it matches an empty sequence the function returns immediately, else each file is passed to the parse function. Finally the result, either an empty list or a list of sequences, are piped to List.concat which flattens the lists.

Wrap-up

All this together gives us a complete csv-file parser, which parses all files in a directory into a single list of objects with properties corresponding to the file headers.

type StatusUpdate = CsvProvider<"statusUpdate.csv">

let parseTransactionFile = 
    (Path.GetFullPath >> StatusUpdate.Load >> (fun (s:StatusUpdate) -> s.Rows |> Seq.toList))

let loadAllStatusUpdates fileDirectory = 
    match Directory.GetFiles(fileDirectory, "*.csv") with
    | [||] -> [||]
    | files -> files |> Array.map parseTransactionFile
    |> List.concat

let statusUpdates = loadAllStatusUpdates "C:\path\to\directory\"

The complete source code of the application can be browsed at https://github.com/spinakr/GlobalMoose

I intend to continue writing posts like this, when I work on or learn something new tech-related. Both to share my experiences going along, but also as a way of documenting what I do. Even if there are thousands of similar articles talking about the same topics, it is very useful to write about stuff when you are learning it. When trying to learn new technology it is, in my opinion, necessary to actually create something that solves a problem, and then present it to somebody. As an consultant you are not always so fortunate that you can chose to work with the newest and coolest technology of your own choosing, thus the need for side projects as a way of evolving as an engineer. In addition to improving my own skill set, I might inspire others to do do the same!