yomimono - something to read

Chamelon: MVP persistent block storage for MirageOS

Fri, 04 Mar 2022 18:00:00 +0000

TL;DR: I wrote a key-value store for MirageOS backed by block storage. It’s called chamelon, it’s based off LittleFS, and if you’re brave, you can use it to store data. Examples are available: a URL shortener and an OAuth2 authorization server.

In English: I couldn’t save or load files before, and now I could. Wowzers!

Faithful High Resolution Textures

Fri, 25 Feb 2022 18:00:00 +0000

I discovered a while ago that some games companies charge extra for “high resolution textures” for their games. This phrase stuck with me until it finally found an outlet: it obviously needs to be rendered in glorious 4-color CGA as a puzzle solution to the 1987 DOS implementation of the popular American game show, “Wheel of Fortune”.

Talks

Fri, 04 Oct 2019 00:00:00 +0000

2019

“Library Operating Systems: Functional Programming for the Whole System”, Haskallywags [slides]

2018

“Library Operating Systems: from Localhost to Remote Guest”, Localhost [slides] [video]

2017

“DHCP: IT’S MOSTLY YELLING!!”, !!Con [slides] [video]
“MirageOS 3: Smaller, lighter, and more transparent” (with Amir Chaudhry), OSCON 2017 [video]
“FuzzOps: keep your computers busy and your pagers quiet”, DevOps Days Madison [slides] [video]
“library operating systems”, 34th Chaos Communication Congress [slides] [video]

2016

“Unikernels and Containers: How to Even”, Open Source Bridge
“Composing Network Operating Systems”, Compose::Conf [slides] [video]
“Unikernels, MirageOS, Library Operating Systems, You”, Madison DevOps [slides]

2015

“Non-Imperative Network Programming”, Strange Loop [slides] [video]
“Persistent Networking with Irmin and MirageOS”, OCaml Users and Developers Workshop [slides] [video]

Ditch That AWS Build Host

Mon, 16 Jul 2018 19:00:00 +0000

In honor of the transnational strike on Amazon this week, here are instructions for moving your AWS unikernels to a cloud that used to claim it wasn’t evil. You might also be interested in establishing a picket line for your packets.

This blog originally ran on Amazon EC2. Since early 2017, it’s been running on a different tech behemoth’s massive public cloud. The deployment process is considerably easier and faster on this alternative public cloud – I first saw it as a live demo given by Michael Bright and immediately knew I wanted to replace my AWS pipeline with it. My AWS unikernel deployments required a secondary Linux host for building AMIs from a kernel image and usually took around 20 minutes from start to finish; GCP deployments can be done from my development host and take around 90 seconds.

Whacking the Bitcoin Piñata

Tue, 17 Apr 2018 18:00:00 +0000

@yomimono or @hannesm surely know if people have tried crowbar on the BTC Piñata. – @kensan@mastodon.social

tl;dr - yes, and it seems that ocaml-x509 is not trivially easy to trick.

Background

The Bitcoin Piñata

In 2015 David Kaloper-Mersinjak and Hannes Mehnert released ocaml-tls, an implementation of TLS (formerly known as SSL) written fully in OCaml. A full writeup of the stack is available in their Usenix Security 2015 paper, and as a series of blog posts on mirage.io. To accompany the release they also deployed a fully-automated bug bounty for the security stack – the bitcoin piñata.

The piñata will establish TLS connections only with endpoints presenting a certificate signed by its own, undisclosed certificate authority, but allows an attacker to easily listen to the encrypted traffic. The piñata always sends the same plaintext in such a connection: the private key to a wallet containing approximately 10 bitcoin. If the attacker can decrypt the ciphertext, or trick the piñata into negotiating a TLS connection with another host and disclosing the key, the information (and therefore the money) is theirs.

Crowbar

Crowbar is a library for writing tests. It combines a property-based API (like QuickCheck) with a coverage-driven generator of test cases (like the fuzzer American Fuzzy Lop). Crowbar tries to find counterexamples to stated properties by prioritizing the generation of test cases which touch more code. It is very good at finding counterexamples.

Testing ocaml-x509

TLS connections are usually authenticated via X509 certificates. ocaml-tls uses ocaml-x509 for this purpose, which is written as a standalone library. There is a clear separation of concerns between ocaml-x509 and ocaml-tls, and a straightforward API for certificate operations in ocaml-x509; both features help tremendously in writing tests for certificate handling.

Posts and Talks Elsewhere

Sat, 23 Dec 2017 18:00:00 +0000

I’ve done a lot of stuff in the last half of 2017, but not much of it has made it here. Here’s a roundup of things published/spoken/embroidered/etc in other places:

NAT your own packets

Mon, 09 Oct 2017 00:03:00 +0000

I’ve been talking about network address translation here for a while, including instructions on building your own NAT device with MirageOS. The library behind those posts, mirage-nat, went on to back talex5’s unikernel firewall for QubesOS, but was unreleased and essentially unmaintained between late 2015 and early 2017.

At the March 2017 MirageOS hack retreat in Marrakesh, talex5 convinced me to do some much-needed maintenance on this library. After having let it age between March and October, I was persuaded to release a version with the hippest latest build system last week. It comes with an example of how you might use it in a MirageOS unikernel that does no additional firewalling. A more compelling example of how you might use mirage-nat (and MirageOS) is still available in qubes-mirage-firewall, which I recommend highly if you’re using QubesOS.

Fine, I'll Download It For You

Tue, 13 Jun 2017 15:14:00 +0000

I recently found myself needing to reset an iPhone 6 to its factory defaults. There is some useful stuff to have if you’re trying to do this:

the passcode for the phone
the AppleID with which the phone is associated
an iTunes installation
an Internet connection capable of downloading the iPhone software image in 15 minutes

It turns out the first two are optional given the last two, or at least a reasonable facsimile. If you don’t have the last one, you have to fake it. Here’s how I spent my first day of funemployment compensating for some Apple engineer failing to consider that it might be nice to download a file before you need to use it.

Crowbar Your Favorite Library for Fun and Bugfixes

Wed, 26 Apr 2017 03:14:49 +0000

Crowbar is a tool that combines afl-persistent’s instrumentation with quickcheck-like property-based testing. afl-fuzz is a great tool for detecting crashes, but Crowbar helps us go a step farther and automatically discover inputs which cause our program to no longer have the properties we expect it to have.

For reasons that don’t need exploring at this juncture, I first thought to apply Crowbar to charrua-client, a library which implements the DHCP state machine from a client perspective. Code snippets below are taken from the dhcp directory in my somerandompacket repository on GitHub.

A Quick Guide to Quick Changes in MirageOS

Thu, 26 Nov 2015 12:14:50 +0000

MirageOS is a collection of libraries and a system for assembling them into unikernels. What happens if you want to make changes to those libraries and test them with a new unikernel?

Say, for example, I have a static website (like this blog) that I build in MirageOS. I want to make some changes to the TCP implementation against which the blog is built. In order to do that, I need to do all the following:

figure out which module to change
figure out which package provides that module
get the source for that package and instruct the package manager to use it instead of the release
make changes
reinstall the package with your changes
rebuild the unikernel completely
see whether changes had the desired effect

Here’s a quick primer on how.

OCaml Workshop and Strange Loop Talks

Wed, 07 Oct 2015 11:05:43 +0000

As a result of great encouragement from colleagues and friends, I gave a few talks in September.

Persistent Networking with Irmin and MirageOS, which I gave at the OCaml Workshop, is a talk on sticking a persistent database into various levels of the network stack. (It includes demonstrations from What a Distributed, Version-Controlled ARP Cache Gets You, as well as an Irmin-ified NAT device that I haven’t yet written up here.) The slides for my OCaml Workshop talk are also available.

Fun with Opam: Advice to my Past Self

Mon, 27 Jul 2015 10:13:10 +0000

Most instructions on how to get started with OCaml packages now advise the user to get started with opam, which is excellent advice. Getting up and running with opam is pretty easy, but I wasn’t sure where to go from there when I wanted to modify other people’s packages and use the modifications in my environment. I wish I’d realized that the documentation for making packages has a lot of applicable advice for that use case, as well as the apparent target (making your own packges from scratch).

Retrospective

Mon, 20 Jul 2015 00:09:32 +0000

In 2014, I spent 12 weeks at the Recurse Center, formerly (and at the time) known as Hacker School. After finishing up my time there in May of that year, a lot of people asked me reasonable questions like:

How was the Recurse Center?
Was attending RC worth your time?
What did you learn at the Recurse Center?

My response to these questions was “I don’t know yet! It’s too early to say.” Now that more than a year has passed, I think I might have some idea of where to start.

What a Distributed, Version-Controlled ARP Cache Gets You

Fri, 24 Apr 2015 16:45:11 +0000

git (and its distributed version control system friends hg and darcs) have some great properties. Not only do you get a full history of changes on objects stored in them, you can get comments on changes, as well as branching and merging, which lets you do intermediate changes without messing up state for other entities which want to work with the repository.

That’s all pretty cool. I actually want that for some of my data structures, come to think of it. Say, for example, a boring ol’ key-value store which can be updated from a few different threads – in this case, a cache that stores values it gets from the network and the querying/timeout code around it. It would be nice if each thread could make a new branch, make its changes, then merge them into the primary branch once it’s done.

It turns out you can totally do that with Irmin, “the database that never forgets”! I did (and am still doing) a bit of work on sticking a modified version of the MirageOS address resolution protocol code’s data structures into Irmin:

$ git log --all --decorate --oneline --graph
* 68216f3 (HEAD, primary, expire_1429688434.645130) Arp.tick: updating to age out old entries
* ec10c9a entry added: 192.168.3.1 -> 02:50:2a:16:6d:01
* 6446cef entry added: 10.20.254.2 -> 02:50:2a:16:6d:01
* 81cfa43 entry added: 10.50.20.22 -> 02:50:2a:16:6d:01
*   4e1e1c7 Arp.tick: merge expiry branch
|\  
| * cd787a0 (expire_1429688374.601896) Arp.tick: updating to age out old entries
* | 8df2ef7 entry added: 10.23.10.1 -> 02:50:2a:16:6d:01
|/  
* 8d11bba Arp.create: Initial empty cache

Let's Play Network Address Translation: The Home Game

Thu, 02 Apr 2015 14:53:00 +0000

When last we spoke, I left you with a teaser about writing your own NAT implementation. iptables (and friends nftables and pf, to be a little less partisan and outdated) provide the interfaces to the kernel modules that implement NAT in many widely-used routers. If we wanted to implement our own in a traditional OS, we’d have to either take a big dive into kernel programming or find a way to manipulate packets at the Ethernet layer in userspace.

But if all we need to do is NAT traffic, why not just build something that only knows how to NAT traffic? I’ve looked at building networked applications on top of (and with) the full network stack provided by the MirageOS library OS a lot, but we can also build lower-level applications with fundamentally the same programming tactics and tools we use to write, for example, DNS resolvers.

Building A Typical Stack From Scratch

Let’s have a look at the ethif-v4 example in the mirage-skeleton example repository. This example unikernel shows how to build a network stack “by hand” from a bunch of different functors, starting from a physical device (provided by config.ml at build time, representing either a Xen backend if you configure with mirage configure --xen or a Unix tuntap backend if you build with mirage configure --unix). I’ve reproduced the network setup bits from the most recent version as of now and annotated them a bit:

Things Routers Do: Network Address Translation

Wed, 01 Apr 2015 11:40:47 +0000

WiFi is fairly ubiquitous in 2015. In most of the nonprofessional contexts in which we use it, it’s provided by a small box that’s plugged into mains power and an Ethernet cable, usually with an antenna or two sticking out of it. I’ve heard these boxes called all kinds of things - hotspots, middleboxes, edge routers, home routers, NAT devices, gateways, and probably a few more I’ve forgotten; there are surely more I haven’t heard. “Router” is the word I hear and use most often myself, despite the unfortunate overlap with a more specific meaning (a device with multiple network links, capable of sending traffic between them). There are an awful lot of things these boxes do which aren’t implied by “router”!

Some Random Idiot

Wed, 22 Oct 2014 17:14:25 +0000

My first interesting job was as a student systems administrator for a fairly heterogenous group of UNIX servers. For the first many months, I was essentially a clever interface to an array of search engines. I came to have a great appreciation for the common phenomenon of a detailed solution to a very specific problem, laid out beautifully in the personal site of someone I’d never met. I answered a lot of “how on Earth did you figure that out?” with “somebody on the Internet wrote about it”.

Virtualization: WTF

Tue, 07 Oct 2014 14:13:03 +0000

For reasons that don’t need exploring at this juncture, I decided to start reading through a bunch of papers on virtualization, and I thought I’d force myself to actually do it by publicly committing to blogging about them.

First on deck is Disco: Running Commodity Operating Systems on Scalable Multiprocessors, a paper from 1997 that itself “brings back an idea popular in the 1970s” – run a small virtualization layer between hardware and multiple virtual machines (referred to in the paper as a virtual machine monitor; “hypervisor” in more modern parlance). Disco was aimed at allowing software to take advantage of new hardware innovations without requiring huge changes in the operating system. I can speculate on a few reasons this paper’s first in the list:

if you have a systems background, most of it is intelligible with some brow-furrowing
it goes into a useful level of detail on the actual work of intercepting, rewriting, and optimizing host operating systems’ access to hardware resources
the authors went on to found VMware, a massively successful virtualization company

I read the paper intending to summarize it for this blog, but I got completely distracted by the paper’s motivation, which I found both interesting and unexpected.

OPW FIN

Fri, 22 Aug 2014 12:56:15 +0000

We’ve come to the end of my round of the Outreach Program for Women, which sponsored my work with the MirageOS folks this summer. I was fortunate to be able to mark the occasion by joining my mentors and an awful lot of badass Xen hackers at the Xen Project Developers Summit earlier this week, where I waved my extremely conspicuous American accent around in everyone’s face and saw some awesome presentations on Xen internals and research. (Xen on ARM is relatively performant! It’s hard to run 10,000 host VMs! The HaLVM has already implemented a whole bunch of stuff I was thinking about doing!)

I Am Unikernel (And So Can You!)

Sun, 17 Aug 2014 23:47:13 +0000

Julia Evans, prolific blogger and rad person, gave me several kind comments on the “Why I Unikernel” posts (security, self-hosting). She also asked, quite reasonably, whether I’d written a high-level summary of how I host my blog from a unikernel. “No, but I should,” I said, and unlike most times I say I should do something, I actually did it.

Here’s the very-high-level overview:

use brain to generate content that some human, somewhere, might want to read (hardest step)
write all that stuff in Markdown
use Octopress to generate a static site from that Markdown
use Mirage to build a unikernel with the blog content
upload the unikernel to an EC2 instance running Linux
build a new EC2 instance from the uploaded unikernel
make sure that newly generated instance looks like my website with new content
shut down the Linux host that made the new EC2 instance
make somerandomidiot.com point to the new EC2 instance
kill the EC2 instance which previously served somerandomidiot.com

And below, one can find the gory details.

My Content is Mine: Why I Unikernel, Part 2

Wed, 13 Aug 2014 15:33:52 +0000

Having a machine capable of executing arbitrary instructions on the public Internet is a responsibility, and it’s a fairly heavy one to assume just to run a blog. Some people solve this by letting someone else take care of it – GitHub, Tumblr, or Medium, for example. I’m not so keen on that solution for a number of reasons, almost none of which are Internet-old-person crankery.

First, and most emotionally: as dumb as my thoughts are, they’re mine. Not GitHub’s or Medium’s or any other group’s. Most entities on the web don’t host user content out of the goodness of their heart; they’re getting something out of it, and it’s likely that they’re getting more out of it than the user is. I’m reminded of the old MetaFilter maxim: “If you’re not paying for it, you’re not the consumer, you’re the product.” Either someone’s making money off of you now or they plan to do it later. I don’t want to encourage that kind of behavior. I just want to write things that people can read about how to make stuff work.

Attack Surface: Why I Unikernel, Part 1

Fri, 08 Aug 2014 14:17:43 +0000

Before I started this blog, I had started a few others at my other domain (now moribund). Despite repeated attempts, I never could resign myself to doing systems administration for a web server that executed dynamic code, like that which powers WordPress or Drupal; I’d install such a framework, begin locking the site down, realize that I’d spent a lot of time reassuring myself that the site was secure without believing it for a second, then delete the framework and revert the frontpage to an index.html rather like what’s present there now. Particularly ambitious iterations would get a post or two published before this cycle completed, now long-vanished.

Doing Nothing in Mirage

Fri, 25 Jul 2014 13:30:49 +0000

It’s Northern Hemisphere summer right now, and in Wisconsin we’re having one of the loveliest ones I can remember. Today the temperature is hovering right at pleasant, there are high clouds blowing across the sky, the breeze is soothing, and birds are singing all over the place. It is not, in short, programming weather. It is sit-outside, read-a-novel, do-nothing weather.

Yes, this sort of thing.

We don’t often let our programs slack off, even when we let ourselves take a peaceful day. I got to wondering (staring off into space, watching the shadows cast by sun-dappled leaves) what the most trivial, do-nothing Mirage project would look like, and how it could be constructed with a minimum of activity and a maximum of understanding.

[] dothraki@iBook:~$ mkdir trivial
[] dothraki@iBook:~$ cd trivial/
[] dothraki@iBook:~/trivial$ ls -alh
total 16K
drwxrwxr-x   2 dothraki dothraki 4.0K Jul 23 13:17 .
drwxr-xr-x 161 dothraki dothraki  12K Jul 23 13:17 ..
[] dothraki@iBook:~/trivial$ mirage configure --xen
[ERROR]      No configuration file config.ml found.
You'll need to create one to let Mirage know what to do.

Okay, we’ll have to do at least one thing to make this work. Mirage uses config.ml to programmatically generate a Makefile and main.ml when you invoke mirage --configure. main.ml uses instructions from config.ml to satisfy module types representing driver requirements for your application, then begins running the threads you requested that it run. That all sounds an awful lot like work; maybe we can get away with not asking for anything.

[] dothraki@iBook:~/trivial$ touch config.ml
[] dothraki@iBook:~/trivial$ mirage configure --xen
Mirage      Using scanned config file: config.ml
Mirage      Processing: /home/dothraki/trivial/config.ml
Mirage      => rm -rf /home/dothraki/trivial/_build/config.*
Mirage      => cd /home/dothraki/trivial && ocamlbuild -use-ocamlfind -tags annot,bin_annot -pkg mirage config.cmxs
empty       Using configuration: /home/dothraki/trivial/config.ml
empty       0 jobs []
empty       => ocamlfind printconf path
empty       Generating: main.ml
empty       Now run 'make depend' to install the package dependencies for this unikernel.
[] dothraki@iBook:~/trivial$ ls
_build  config.ml  empty.xl  log  main.ml  Makefile

That seems like a great start! Maybe we can trivially achieve our dream of doing nothing.

[] dothraki@iBook:~/trivial$ make depend
opam install mirage-xen --verbose
[NOTE] Package mirage-xen is already installed (current version is 1.1.1).

Resting on our laurels. Excellent. (In keeping with the lazy theme of this post, I’ll elide the make depend step from future examples, but if you’re playing along at home you may discover that you need to run it when you introduce new complexity in pursuit of perfect non-action.)

[] dothraki@iBook:~/trivial$ make
ocamlbuild -classic-display -use-ocamlfind -pkgs lwt.syntax,mirage-types.lwt -tags "syntax(camlp4o),annot,bin_annot,strict_sequence,principal" -cflag -g -lflags -g,-linkpkg,-dontlink,unix main.native.o
ocamlfind ocamldep -package mirage-types.lwt -package lwt.syntax -syntax camlp4o -modules main.ml > main.ml.depends
ocamlfind ocamlc -c -g -annot -bin-annot -principal -strict-sequence -package mirage-types.lwt -package lwt.syntax -syntax camlp4o -o main.cmo main.ml
+ ocamlfind ocamlc -c -g -annot -bin-annot -principal -strict-sequence -package mirage-types.lwt -package lwt.syntax -syntax camlp4o -o main.cmo main.ml
File "main.ml", line 8, characters 2-13:
Error: Unbound module OS
Command exited with code 2.
make: *** [main.native.o] Error 10
[] dothraki@iBook:~/trivial$

Oh, bother.

How to Set the Evil Bit

Tue, 08 Jul 2014 15:23:55 +0000

Our mission: fuzzing TCP options from scapy.

Our target: the echo service from mirage-tcpip/examples/services.ml.

Outcome: a revision on a widely-used OCaml dependency, gleeful murder and resurrection of several EC2 instances, something to brag to my mom about, a look at a case worse than failure, and great justice.

Parsers Optional

Mon, 07 Jul 2014 12:49:38 +0000

Friends, I have spoken to you of TCP and of fuzzing. Next I will speak to you of both, but today, I will speak to you of TCP options. If you’re here for the pwnage, sit tight; it’s coming.

What Even Is TCP Anyway

Here’s the lazy way of explaining it: TCP is the abstraction layer that allows you to pretend that network communication works in a logical, orderly, reliable fashion when you’re writing an application. Reading data and having it always be in the order it was sent? TCP. Being able to know whether a connection is open or closed? TCP. Knowing the difference between data coming from two separate processes on the same remote host? TCP. (There are other ways to get these guarantees, but the vast majority of Internet traffic that needs them gets them via TCP.)

On a less abstract level, TCP is a header (one of several!) that your operating system slaps on your network traffic before shipping it over the wire, on the way to its final destination. For damn near all the information on TCP you can shake a stick at, you can consult RFC 793 directly. The header summary, most relevant for our exploration, is reproduced below:

0                   1                   2                   3   
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |       Destination Port        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Sequence Number                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Acknowledgment Number                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Data |           |U|A|P|R|S|F|                               |
| Offset| Reserved  |R|C|S|S|Y|I|            Window             |
|       |           |G|K|H|T|N|N|                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Checksum            |         Urgent Pointer        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Options                    |    Padding    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                             data                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Everything here is a fixed-length field except for Options, Padding, and data, all of which are optional. Data is up to the application, when it’s present (and is also frequently referred to as payload). When you loaded this web page, TCP packets were sent from my server at somerandomidiot.com to your computer, and the contents of the data field were these very words that you’re reading right now. TCP is data-agnostic; it only cares that your payload arrives intact, not what’s in it.

Options, on the other hand, are very much TCP’s concern.

Hacking Your Hacker School T-Shirt

Wed, 25 Jun 2014 16:01:55 +0000

On the last day of our Hacker School batch, we got some cool commemorative T-shirts with the stylish Hacker School logo on them. Now, this is an excellent T-shirt, and the only way I know of to get an official one is to attend the last day of a batch of Hacker School, so it’s already a pretty rad shirt. I wanted to make it radder, and I did, and now I want to tell you how.

The Minnesota Goodbye

Tue, 03 Jun 2014 13:28:43 +0000

Looking into some of the results from last week’s fuzzing session, I noticed something interesting:

$ tcpdump -r experimenting_with_pathoc.pcap 'src host 192.168.2.24 and tcp[13] & 1 != 0'
reading from file experimenting_with_pathoc.pcap, link-type EN10MB (Ethernet)
$

Let’s translate that into human.

tcpdump -r experimenting_with_pathoc.pcap: use tcpdump to read an existing packet trace named experimenting_with_pathoc.pcap.
src host 192.168.2.24: show me only packets that were sent by 192.168.2.24, which is the IP address of a running unikernel that’s serving web pages on port 80.
and tcp[13] & 1 != 0: of the packets sent by 192.168.2.24, show me only those where the least significant bit of the 13th byte of the TCP header was not zero. The 13th byte of the TCP header is designated for flags relevant to how the packet should be processed by the TCP state machine, and the least significant bit corresponds to the FIN flag, used to initiate graceful connection closures.

All together, “show me all the packets sent by 192.168.2.24 which initiated a graceful connection closure.” tcpdump helpfully shows us… all zero such packets in the trace.

This isn’t necessarily wrong for a webserver implementing HTTP/1.1, which defaults to persistent connections:

8.1.2 Overall Operation

A significant difference between HTTP/1.1 and earlier versions of HTTP is that persistent connections are the default behavior of any HTTP connection. That is, unless otherwise indicated, the client SHOULD assume that the server will maintain a persistent connection, even after error responses from the server. – RFC 2616

So let’s make something that will try to initiate a connection closure.

Throwing Some Fuzzy Dice

Thu, 22 May 2014 10:42:56 +0000

I mentioned a while ago that the Mirage project agreed to have me on board, through the OPW internship project, for the summer. We started on Monday, and I’ve already had a lot of fun!

Officially, my job for the summer is to help shore up the network stack in Mirage, in part by running the current code through its paces, and in part through implementing some new functionality. This first week, I continued some work I started at the end of Hacker School - figuring out how to fuzz some strange (and not-so-strange) corners, and how to wrangle the data I got out of doing so.

Fuzz What Now?

Let’s step back. Way, way, way back.

If you’re a computer program, and you have some data that you care about, your data is likely in some kind of structure reflecting an underlying order to that data. Objects are a common way to organize this stuff; dictionaries, hashmaps, lists, arrays, trees, the list goes on. That’s all well and good when your program is running, keeping all this stuff in memory. But it happens depressingly often that you need to dump this stuff to permanent storage, or express it in some way to some other program or another computer, or represent it on the screen because something awful has happened, and you can’t just say “memory address 0x52413abd, memory address 0x52413cda, memory address 0x52413ea2” - these things are meaningless outside the context of the current run of that program.

So we have serialization, the high-level concept for the jillion different ways to take that data and put it in a string, or a binary data format, so something else can read that string and reassemble the structure of the data. That’s deserialization, which implies parsing; parsing is a pretty big deal.

When the data you’re attempting to assemble into a structure is as you expect it and everything is correct, parsing’s no problem. But it frequently happens that everything is not as you expect it, for any number of reasons - the programmer who made the program that made the message made a mistake; the programmer who made the program that reads the message made a mistake; the programs reading and writing the message are using different versions of the specification in the first place; the specification wasn’t specific about whether the third byte’s range from 0 to 5 was inclusive or exclusive and each programmer made a different decision; both programs agree, but the message was corrupted in transit; the message was corrupted in transit, and one program has implemented a different corruption recovery algorithm than the other… I’ll stop now, but I could keep going for a long time.

There are a lot of bad messages out there. It’s hard to make your parser do the right thing when it receives an arbitrary bad message. It can be hard to even know that your parser does the wrong thing when it receives an arbitrary bad message - if you thought of a certain kind of bad message to use in testing, of course you fixed your code to deal with it; you thought of it! But there are almost certainly loads more bad messages out there than the ones you thought of - both by chance, and by design.

If humans can’t make enough bad messages, maybe computers can. Randomly generating a whole mess of bad messages, sending them to your program, and seeing what happens is called fuzz testing, and it’s awesome.

Verb Your Own Noun

Wed, 23 Apr 2014 12:03:04 +0000

This blog has been running on a Mirage OS unikernel hosted on Amazon EC2 since April 3rd:

$ ec2-get-console-output --region the-best-region i-0123abcd
2014-04-03T16:42:58+0000
Xen Minimal OS!

In that time, I’ve done some stuff:

submitted a successful pull request to Mirage
made a permanent home for Secret Project Glow Cloud’s code
successfully applied to the Outreach Program for Women to work on Mirage some more
broke and fixed my OCaml development environment repeatedly
sang in public in front of other Hacker Schoolers
looked at a lot of neat objects and buildings
started work on a fuzzing framework to scratch my own itch for testing network clients

I figured it was time to tell you about some of it, but first I did some other stuff:

upgraded some packages on my build machine
broke the build on my blog
learned about how Mirage makefiles are generated by trying to get mine working again

You’d rather hear about all of that, right?

Tying the Knot

Wed, 02 Apr 2014 14:58:53 +0000

“This is a pretty strange piece of code, and it may take a few moments of thought to figure out what’s going on.”

– Real World OCaml

A few weeks ago, fellow Hacker Schooler Chen Lin and I were trying to solve a simple graph problem in Haskell. I was all ready to charge forward with something quite like the Java implementation I learned back in undergrad, but my fellow Hacker Schooler had some hesitation around whether this kind of structure would work in Haskell.

After a little bit of Googling, I found out that the canonical solution in Haskell involves something intriguingly dubbed tying the knot. I stared blankly at this HaskellWiki page with my fellow Hacker Schooler, trying to understand it quickly enough to have a useful conversation about it, and failed. We threw a couple of other ideas around and then decided to both pursue other projects. I moved on, Chen moved on, and I’m not sure either of us thought much about it…

…until yesterday, when I ran into tying the knot again. This time, it was hiding deep within (of all things!) the chapter on imperative programming in Real World OCaml, and I was unhurried and determined. “Abstract concept, I am going to understand you so hard,” I thought, jaw set.

Arriving At the Mirage

Mon, 24 Mar 2014 17:57:37 +0000

When last we left our hero, I was strugging valiantly to get a Mirage unikernel version of this blog running on Amazon EC2. All unikernels built and shipped off to EC2 would begin booting, but never become pingable or reachable on TCP port 80. ec2-get-console-output on any instance running a Mirage unikernel would show the beginning stages of a DHCP transaction, then the disappointing RX exn Invalid_argument("String.sub"), then… silence.

When all you had for many years was a hammer, stuff is still going to look an awful lot like nails to you, even if it’s pretty distinctly screw-shaped. I wanted to take a packet trace of this transaction pretty badly. I could do three things that were almost like this:

get a packet trace of another machine getting a DHCP lease on EC2
get a packet trace of a unikernel getting a DHCP lease on my local Xen server
print out an awful lot of diagnostic data from the EC2 unikernel and read it from the console

Trying to draw some conclusions from the first option above led me down the wrong path for about a day or so. I did manage to cause the DHCP client to fail on my local Xen server by sending a DHCP reply packet with no server-identifier set, using scapy and some hackery to cause the xid to always match:

Advancing Toward the Mirage

Sun, 16 Mar 2014 15:11:09 +0000

I left off last time telling you about getting Mirage to not work. I’m still working hard to get this blog – yes, this one you’re reading now – up and running as a unikernel on EC2.

It became clear to me last week that I needed to fork my own instance of the mirage-tcpip repository and compile my kernels with it, if I were to make any progress in debugging the DHCP problems I was having. A few naive attempts to monkey with version of mirage-tcpip downloaded by opam weren’t successful, so I set about to figure out how actual OCaml developers develop in OCaml with opam.

First stop: the opam documentation on doing tricky things. This is a little short of a step-by-step “do this, dorp” guide, unfortunately; here’s what I end up doing, and it sorta seems to work.

It's a mirage! (Or, how to shave a yak.)

Fri, 14 Mar 2014 15:53:00 +0000

A week or so ago, I heard about the Mirage project, a library OS project that makes tiny virtual machines running on top of Xen to run a given application, and do nothing else. I was intrigued, and started working through the excellent intro documentation, and got to the point where I wanted to replace my ho-hum statically-compiled blog hosted from Ubuntu LTS with a unikernel that would serve my static site and do nothing else.

There are excellent instructions on doing this with a Jekyll site on Amir Chaudhry’s blog. Octopress, which I use to generate this site, is built on top of Jekyll, and I only had a few extra goodies to throw in before I was able to make a unikernel that would run my blog with a few rake invocations. After getting the first unikernel up and running via Xen on my laptop, I entertained myself by throwing a few nmap commands at it; I was particularly curious to see whether my unikernel knew what to do with UDP packets:

sudo nmap -sO 192.168.2.13

Starting Nmap 6.40 ( http://nmap.org ) at 2014-03-14 23:26 EDT
Nmap scan report for 192.168.2.13
Host is up (0.00037s latency).
Not shown: 254 open|filtered protocols
PROTOCOL STATE SERVICE
1        open  icmp
6        open  tcp
MAC Address: 00:16:3E:53:E0:1B (Xensource)

Nmap done: 1 IP address (1 host up) scanned in 17.72 seconds

Hee hee hee.

Finding Kitten

Sun, 09 Mar 2014 16:57:45 +0000

Robot Finds Kitten

Sometime way back in the past, a human who wanted me to feel joy introduced me to Robot Finds Kitten, a Zen simulation which is pretty close to exactly what it says on the tin. There are already quite a lot of ports of the original POSIX implementation, but none of them were written in Elm. Obviously this is a problem that needs fixing.

Before I get into the gory details of learning Elm via robots, I should tell you that my implementation is available for free play (edit: sorry, this has bitrotted too much to be included anymore), and you can also go look at the source code.

Elm

I got a really wonderful introduction to Elm when Evan Czaplicki came to Hacker School in our second week. We got a slightly adapted version of this talk from StrangeLoop 2013, which moved me to make a browser game (something I’ve never wanted to do at any previous point in life). The language seemed elegant and expressive, for lack of less cliched words, and I thought it might be relatively simple to make a succinct Robot Finds Kitten clone.

Sometimes it's no fun to be right.

Tue, 25 Feb 2014 19:05:30 +0000

I promised a lot of people that I would let them know how Hacker School is. It’s difficult for me to answer this question (although Julia Evans, a previous batch process, has done a fantastic job), both because it feels so early in the batch and because Hacker School is a lot of things. I also promised a lot of people that I would let them know how New York is, and that’s a little easier, so I’ll start there and then move on.

Early Thoughts on Hacker School

Wed, 12 Feb 2014 03:47:52 +0000

I got my acceptance notification for the winter 2014 batch of Hacker School on January 3rd, six weeks ago. Right after being accepted, I wrote a bit in the same directory where I’d saved my application answers:

Secret Project Glow Cloud (Embroidery)

Wed, 22 Jan 2014 01:56:56 +0000

All of the stitches in the wearable circuit elements of Secret Project Glow Cloud were simple running stitches, which I tried to keep as consistent as possible. The real embroidery work for the Glow Cloud was concentrated in two decorative elements: the Welcome to Night Vale logo in front of the light sensor, and the cloud design in front of the light circuit.

Secret Project Glow Cloud (Electronics)

Sat, 18 Jan 2014 18:42:41 +0000

Glow Selfie

A few months ago I ordered this Glow Gloud T-shirt, one of the first pieces of merchandise available promoting Welcome to Night Vale. A lot of other people love Welcome to Night Vale, and so the shirts were massively oversold for their first printing; as a result, I wasn’t sure when I was going to get my shirt. With a determination born of despondency, I turned to a big box of stuff I had ordered from Adafruit and a pile of fabric, and I said “Okay, I’ll make my own T-shirt. With glowy bits! And I’ll bet I can get it done before this T-shirt everyone else is going to have arrives!”

Four months later (and about three and a half months after the mass-produced T-shirt arrived), I wore my shirt for the first time. Here’s how I made it.