December 13 2022, 00:22

I finished reading “Endurance” by Alfred Lansing. A very interesting story, it’s a page-turner. The only thing is that you gradually need to get accustomed to the vocabulary, which is unusual for an IT person far removed from the sea.

And to illustrate, look at what a programmer does in about 20 minutes. He downloads the book in EPUB format (found only via Yandex on some pirate sites). Unzips EPUB to HTML. Writes a quick Python script that extracts text from the HTML, breaks the text into sentences, the sentences into words, and converts the words into their normal form (using NLTK). Downloads a word frequency corpus from Kaggle — 333K words with usage frequencies. Then, for each sentence, it calculates two values — the number of words and the number of rare words. Then it sorts the sentences that are at least 10 words long and have a frequency below a certain threshold (500K). Eventually, one of the top results is a sentence that I even remember reading in the book and was just thinking about as something to post on Facebook to illustrate the complexity of the text:

“…A vulgar person who often indulges in whoops and yells of ‘Yoicks Tally Ho,’ had the indescribable effrontery to let go his horrid war cry whilst riding on the imposing conveyance drawn by these dignified but nervous creatures, and was reproved by their indignant owner pointing out to the Vulgar Person into what terror his voice had thrown the beautiful but highly strung and delicate doggies. It is my painful duty to relate that this Awful Vulgar Person the very next day being out with an ordinary team gave vent to his fearsome bellow when passing the ‘Heart Disease’ Team. The result was disastrous, 2 of the poor creatures fainted and had to be brought round with hartshorn, etc., while the remainder went into hysterics until the Vulgar Person and his associates disappeared over the horizon.”

But it must be said that such complex text is not encountered very often. These are excerpts from the diary of Frank Worsley. This sailor expresses his thoughts in a rather unusual way for someone reading his words a century later. The book itself, written from the perspective of a well-educated 20th-century person and journalist, is relatively easy to read (again, one needs to get used to the vocabulary, some nautical terms will have to be googled).

Leave a comment