Posts

Just a quick list of thoughts I have about system stability. Alas, I have little data supporting these conjectures, all I have is distilled experience. Systems have inertia. A stable system tends to remain stable until destabilized by a change. Once destabilized, a system may be more unstable for a period of time. Some systems have more inertia than others Simple systems are more robust to changes than others. If you have a sufficiently complex production environment (one involving a plethora of legacy systems), sometimes one might stumble upon some old process happily chugging along on a legacy server....

I know many of these statistics are out there on the internet, but searching for them takes time, so this post collects a bunch of them in one place (likely just for my future consumption). English Language Statistics Peter Norvig has a list of the 100,000 most frequent english words derived from the “Google Web Trillion Word Corpus” 1. English Word Length Relative Frequencies I like relative frequencies, so here are some histograms showing english word length and relative frequency....

Regression toward the mean is a statistical phenomena that, once you notice it, you cannot help but notice how surprisingly ubiquitous it is. Coin Flip Competitions OK, so what is regression toward the mean? It’s probably best to start with an example where common sense intuitions will not strongly deceive you into believing a non-statistical explanation, so we are going to turn to something which might only be observed in primary school playgrounds: coin-flipping contests....

I’m far from the world’s biggest advocate of AWS lambda (or serverless computing more broadly), but I have used it for a few production applications. This post will hopefully be a living document of all the times choosing a serverless computing environment has caused me great pain 1. Caching outside /tmp Many libraries have assets which are downloaded at runtime. It may not be obvious to you that this is the case, however, until your run into the problem of “said library downloading files outside of /tmp on a lambda” function....

An overly simple model of success might be: Opportunities present themselves according to some distribution 1, with the payout from an opportunity largely a function of the opportunity’s likelihood (the rarer, the higher the payout). An individual can take on an opportunity if they are able to process it within a certain time 2. Success is the sum of all opportunities. Due to the long-tailed nature of opportunities, an individual’s success is, to a significant degree, due to rare but impactful opportunities....

It seems likely that the world would be a marginally better place if people and organisations abandoned underperforming projects more often. Underperforming projects are a time suck, consuming resources which could be better allocated to other projects. If you feel uncertain about the value of a personal project or are finding it taking longer than expected, it might be worth abandoning it. If you notice a work project is struggling to reach completion or is delivering relatively poor returns, it might be worth speaking up and asking whether the project is worth it....

Recently, I wanted to access some data stored in a public google cloud storage (GCS) bucket. I could access the data by authenticating, but it seemed desirable to see how else I could access the data. Google’s documentation suggests that it’s possible to access the data over https, with all public bucket objects accessible via the following url with BUCKET_NAME and OBJECT_NAME appropriately substituted. https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME The above, however, is only useful if you already know the object you are interested in....

Something that seems to hold true across all organisation sizes is that (at least a sizeable subset of) software developers are opinionated (with perhaps the number of opinionated developers increasing with scale). Opinions exert a considerable influence on software development practices, with many developers interested in “what are the best practices”. After many decades as a profession, it seems fair to say that there is now a reasonable base set of best practices that all software development teams should adopt....

I wasn’t going to bother writing this up (as I hoped the documentation had improved), but it’s surprisingly difficult to get the non-burstable network bandwidth for an EC2 instance from the official AWS documentation. Instead, you are always quoted the burst values, which can be relatively uninformative. To get the actual bandwidth, here are two options: A rough heuristic is that every “downsize” corresponds to a halving of base network bandwidth....

As mentioned elsewhere, this website uses Hugo with the PaperMod theme. Although I have not put any significant amount of effort into making it so this website is indexed by search engines, I have put a small amount of effort. This documents what I have done. Meta Tags If you are interested in making your website more easily accessed by search engines, it’s important that the website has a description meta tag....