Home » Archives for Jesse Davis

Author Archives: Jesse Davis

Jesse is a senior engineer at MongoDB in New York City. He specializes in Python, MongoDB drivers, and asynchronous frameworks.

How To Deadlock Your Python With getaddrinfo

What happens if you run this code? import os import socket import threading def lookup(): socket.getaddrinfo('python.org', 80) t = threading.Thread(target=lookup) t.start() if os.fork(): # Parent waits for child. os.wait() else: # Child hangs here. socket.getaddrinfo('mongodb.org', 80) On Linux, it completes in milliseconds. On Mac, it usually hangs. Why? Journey To The Center Of The Interpreter Anna Herlihy and I tackled ...

Read More »

Black Pipe Testing A Connected Application In C

This is the fourth article in my series on “black pipe” testing. Here I describe testing libmongoc (the MongoDB C Driver) as a black pipe. Like any network client library, libmongoc cannot be fully tested as a black box. Traditional black box tests enter some input and check the output—this only validates one side of the system at a time. ...

Read More »

Test MongoDB Failure Scenarios With MockupDB

This is the fifth article in my series on “black pipe” testing. Traditional black box tests work well if your application takes inputs and returns output through one interface: the API. But connected applications have two interfaces: both the API and the messages they send and receive on the network. I call the validation of both ends a black pipe ...

Read More »

MockupDB: Testing The MongoDB C Driver With Python

This is the third article in my series on “black pipe” testing. The series describes how to test networked code, like a MongoDB driver. Such code has two distinct public surfaces: one is its API, and the other is its communication over the network. Treating it as a black box tests just one surface, the API. To test both, we ...

Read More »

Testing PyMongo As A Black Pipe

This is the second article in my six-part series on “black pipe” testing. PyMongo, the official Python client for MongoDB, is a great example of a connected application that can’t be fully tested as a black box. It has two ends that take inputs and provide outputs: one is its public API, the methods find and insert_one and so on. ...

Read More »

The MongoDB Shell’s New CRUD API

We released the latest development version of MongoDB the other week. The official announcement focuses on bug fixes, but I’m much more excited about a new feature: the mongo shell includes the new CRUD API! In addition to the old insert, update, and remove, the shell now supports insertMany, replaceOne, and a variety of other new methods. Why do I ...

Read More »

Eventually Correct: Async Testing With Tornado

Async frameworks like Tornado scramble our usual unittest strategies: how can you validate the outcome when you do not know when to expect it? Tornado ships with a tornado.testing module that provides two solutions: the wait / stop pattern, and gen_test.   Wait / Stop gen_test Further Study Wait / Stop To begin, let us say we are writing an ...

Read More »

PyMongo And Key Order In Subdocuments

Or, "Why does my query work in the shell but not PyMongo?" Variations on this question account for a large portion of the Stack Overflow questions I see about PyMongo, so let me explain once for all. MongoDB stores documents in a binary format called BSON. Key-value pairs in a BSON document can have any order (except that _id is ...

Read More »