Trailblazing Artisans of Road Building

A lot of us take roads for granted, at least until they are icy or torn up by construction. The concept of creating fixed paths seems to be in our firmware. Finding the shortest distance to food or water and marking a trail to it from home base has always been its own reward.

Roads have come a long way from the simple paths beaten by man and beast. But the basic configuration of paved roads hasn’t changed all that much since the Roman empire. Whatever they’re made of, they need to be able to drain water and support heavy loads.

New issues arose as modes of transportation shifted in favor of the automobile. Road surfaces needed to provide friction against tires. But how did we get from the stone-paved roads of Rome to the asphalt and concrete roads of today?

Continue reading “Trailblazing Artisans of Road Building”

How Hacker News Page Rankings Really Work

Page rankings are the secret sauce of websites that automatically aggregate user submissions. The basic formula used by Hacker News was published a few years back. But there are several pieces of the puzzle that are missing from that specification. [Ken Shirriff] recently published an analysis that digs deeper to expose the article penalization system used by Hacker News’ ranking engine.

One might assume that the user up and down votes are what determine a page’s lifespan on the front page. But it turns out that a complex penalization system makes a huge difference. It takes into account keywords, and domain names but also weighs controversy. It’s a bit amusing to note that this article on the topic was itself penalized, knocking it off of the front page.

You can get the full details of the system from his post, but we found his investigation methods to be equally interesting. He scraped two pages of the news feed every minute using Python and the Beautiful Soup package (a pretty common scraping practice). This data set allowed him to compare the known algorithm with actual results. What was left were a set of anomalies that contained enough sense for him to reverse engineer the unpublished formulas being used.