Accelerate Your Development with FsHttp and FSharp.Data

Ditch cURL and Postman for readable HTTP queries and a parser that understands your data before you write a line of code

So you have to hit a new JSON API. Is there an SDK library? Nope.

Forgot what flag you need for cURL? Time to use Postman. Postman asking you to sign in? Hmm... maybe you want a notebook experience? pip install requests but your Python installation is broken again....

You think to yourself,

there has to be a better way!

Well, fortunately, there is. And today I'm going to help you set it up in a matter of minutes.

Installation 🧑‍💻

.NET runs on Mac?!

First, you need to install dot.net.

Now, I maybe can guess what you might be thinking.

Another SDK? Just for glorified cURL?

Maybe the best part of .NET is the developer experience. It's on your favorite package manager, there aren't any ".NET version managers" - you just install the version you want and it works, and you install tools globally, not packages, so you couldn't even break your installation just by installing packages if you tried.

Polyglot Notebooks

The .NET SDK already comes with all the F# tools we need to run these commands in the terminal with fsi, but this experience works much better with Polyglot Notebooks over VSCode with code --install-extension ms-dotnettools.dotnet-interactive-vscode.

Syntax Sugar 🍬

The first package I'd like to show is:

#r "nuget: FsHttp"
open FsHttp

Postman is useful because we - as humans - don't always remember the exact names of all the headers of the HTTP specification. But we kinda do at least know what a HTTP request is supposed to look like.

FsHttp uses an F# feature called computation expressions (see builder pattern, monad) to wrap System.Net.Http so it actually looks like an HTTP request.

http {
    GET "https://en.wiktionary.org/w/api.php?action=query&titles=%E5%AE%9D&prop=revisions&rvprop=content&rvsection=9&format=json"
}
|> Request.send
|> Response.toFormattedText

{
  "batchcomplete": "",
  "warnings": {
    "main": {
      "*": "Subscribe to the mediawiki-api-announce mailing list at \u003Chttps://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/\u003E for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application."
    },
    "revisions": {
      "*": "Because \u0022rvslots\u0022 was not specified, a legacy format has been used for the output. This format is deprecated, and in the future the new format will always be used."
    }
  },
  "query": {
    "pages": {
      "14156": {
        "pageid": 14156,
        "ns": 0,
        "title": "\u5B9D",
        "revisions": [
          {
            "contentformat": "text/x-wiki",
            "contentmodel": "wikitext",
            "*": "==Japanese==\n{{ja-kanji forms|\u5B9D|\u5BF6}}\n\n===Kanji===\n{{ja-kanji|grade=6|rs=\u5B8005|kyu=\u5BF6}}\n\n# [[precious]] [[objects]]\n# [[worldly]] [[goods]]\n# [[valuable]] [[possessions]]\n\n====Readings====\n{{ja-readings\n|goon=\u307B\u3046\n|kanon=\u307B\u3046\n|kanyoon=\u307B\n|kun=\u305F\u304B\u3089-\n}}\n\n===Etymology===\n{{ja-kanjitab|\u305F\u304B\u3089|yomi=k|alt=\u8CA1,\u8CA8}}\n\nFrom {{inherited|ja|ojp|-|sort=\u305F\u304B\u3089}}. First cited to the \u0027\u0027{{w|Man\u0027y\u014Dsh\u016B}}\u0027\u0027 of 759 {{CE}}.\u003Cref\u003E{{R:Nihon Kokugo Daijiten 2|\u5B9D\u30FB\u8CA1\u30FB\u8CA8}}\u003C/ref\u003E From {{inh|ja|jpx-pro|*takara}}.\n\nSamuel Martin analyzes this as a compound of {{m|ja|\u9AD8|tr=taka-|t=high}} \u002B {{m|ja|\u7B49|tr=-ra|pos=pluralizing suffix}}.\u003Cref\u003E{{R:ja:Martin 1987}}\u003C/ref\u003E  However, this is semantically problematic, as such a compound would ordinarily refer to \u0022the [[heights]]\u0022 as a location, and there is no clear means of deriving the sense of \u0022[[treasure]]\u0022 from the proposed component parts.\n\nSome sources derive this as a compound of {{com|ja|\u7530|tr1=ta|t1=paddy field|\u304B\u3089|tr2=kara|t2=from|lit=from the paddy fields}}, from the way people value thriving paddy fields as a unique kind of treasure.\u003Cref\u003E[https://www.takarashuzo.co.jp/takarahatakara/ \u5B9D\u306F\u7530\u304B\u3089\uFF5E\u79C1\u305F\u3061\u306E\u539F\u70B9\uFF5E | \u5B9D\u9152\u9020\u682A\u5F0F\u4F1A\u793E] - Treasure is from the rice fields -Our Beginning- | Takara Shuzo (In Japanese)\u003C/ref\u003E  However, the sense of \u0022from\u0022 for {{ja-r|\u304B\u3089}} does not appear until roughly the {{w|Heian period}},\u003Cref\u003E{{R:Nihon Kokugo Daijiten 2|\u304B\u3089}}\u003C/ref\u003E more recent than the first appearance of \u0027\u0027takara\u0027\u0027, making this a [[folk etymology]].\n\n===Pronunciation===\n{{ja-pron|\u305F\u304B\u3089|acc=3|acc_ref=DJR,SMK5,NHK}}\n\n===Noun===\n{{ja-noun|\u305F\u304B\u3089}}\n\n# {{defdate|from 759}} [[treasure]]\n\n===References===\n\u003Creferences/\u003E"
          }
        ]
      }
    }
  }
}

Postman's UI makes it ease to identify the headers you need, but Postman is less trivial to edit and document with (see Postman collections).

FsHttp requests stored in fsi/ipynb files are just that - files, so they're dead simple to edit and document, but IntelliSense makes it trivial to write the exact request you want. Fun!

HTTP files can be a great alternative to Postman. Simply put - HTTP files store HTTP requests in text form, and your editor tooling can help you fill out the appropriate request and execute it.

You can even use HTTP files in the form of HttpRepl blocks in a polyglot notebook!

Start-ThreadJob -ScriptBlock {
    # Define the URL and path
    $url = "http://localhost:8080/foo"
    # Define the content to write
    $content = "So a foo walks into a bar..."
    # Create an HttpListener
    $listener = [System.Net.HttpListener]::new()
    $listener.Prefixes.Add("http://localhost:8080/")
    # Start the listener
    $listener.Start()
    Write-Host "Listening on http://localhost:8080/"
    $context = $listener.GetContext()
    $response = $context.Response
    $buffer = [System.Text.Encoding]::UTF8.GetBytes($content)
    $response.ContentLength64 = $buffer.Length
    $response.OutputStream.Write($buffer, 0, $buffer.Length)
    $response.OutputStream.Close()
}
| Out-Null

GET http://localhost:8080

Stop-Job -Id 1
Remove-Job -Id 1

So far, we haven't found a strong reason to use FsHttp over HTTP files, but there's one big advantage FsHttp has over HTTP files: programmability.

Data as First-Class Citizens 🗳️

If you're familiar with Python or JavaScript, you might be used to dot-navigating your way through API responses but having to do data validation manually. If you've used C#, you're probably used to getting a lot of data validation for cheap with JsonDeserialize, while still having to define your schema up front with records and classes.

With F# type providers, you can actually get the benefit of both of these with neither of the costs. Let me show you what I mean:

#r "nuget: FSharp.Data"
open FSharp.Data

type GitHubRepositories = JsonProvider<
"""
{
  "items": [
    {
      "id": 29048891,
      "name": "fsharp",
      "full_name": "dotnet/fsharp"
    }
  ]
}
""">
GitHubRepositories.Load("https://api.github.com/search/repositories?q=language:fsharp&per_page=5")
  .Items
  |> Array.map (_.FullName)

[ dotnet/fsharp, fable-compiler/Fable, fsharp/fsharp, giraffe-fsharp/Giraffe, fsprojects/Paket ]

Above, we gave JsonProvider an inline sample of our schema. This sample got ingested by the build process and was used to generate the rest of the GitHubRepositories type.

Passing in hard-coded or file referenced samples like this is recommended in real applications since the data becomes part of the build process. However, for just experimenting in an interactive session, it's perfectly fine to directly pass an API endpoint to JsonProvider.

type GitHubRepositories = JsonProvider<"https://api.github.com/search/repositories?q=language:fsharp&per_page=5">
GitHubRepositories.GetSample()
    .Items
    |> Array.map (fun x -> x.FullName, x.StargazersCount)

dotnet/fsharp

3862

fable-compiler/Fable

2887

fsharp/fsharp

2172

giraffe-fsharp/Giraffe

2106

fsprojects/Paket

2014

Because FsHttp is a wrapper for building requests and FSharp.Data is a library for intepreting (well, data generally, but in our case) responses, and because .NET has an awesome "work together" culture instead of "rebuild a worse version yourself 'from ground up'" culture, these two libraries work together rather flawlessly ✨:

type Kanji = JsonProvider<"https://kanjiapi.dev/v1/kanji/力">
let stream =
    http {
        GET "https://kanjiapi.dev/v1/kanji/生"
    }
    |> Request.send
    |> Response.toStream
let 生 = Kanji.Load(stream)
Array.concat (seq { 生.KunReadings; 生.OnReadings; })

[ -う, い.かす, い.きる, い.ける, う.まれ, う.まれる, う.む, うま.れる, うまれ, お.う, き, な.す, な.る, なま, なま-, は.える, は.やす, む.す, ショウ, セイ ]

Next Steps 🤔

Don't wait! If a lot of things in this blog post seemed unfamiliar, you will assume they're hard and have already created a mental blockade. You will need an HTTP client later, but the energy to set up FsHttp will seem greater than dealing with the overhead of your Least Common Denominator tool, but over time that overhead will pile up and contribute to developer burn out.

Refresh your toolkit today! Start installing dot.net, take a break from the computer screen and stretch those legs, then in the browser or !

Happy coding!