Tag: TechStories

…On Trump’s first day as president, Musk went to the White House to be part of a roundtable of top CEOs, and he returned two weeks later for a similar session. He concluded that Trump as president was no different than he was as a candidate. The buffoonery was not just an act. “Trump might be one of the world’s best bullshitters ever,” he says. “Like my dad. Bullshitting can sometimes baffle the brain. If you just think of Trump as sort of a con-man performance, then his behavior sort of makes sense.” When the president pulled the U.S. out of the Paris Accord, an international agreement to fight climate change, Musk resigned from the presidential councils.

Today, I am delving into ML algorithms and was surprised to learn that the numpy library used to depend on Fortran code (BLAS/LAPACK) until recently, but now checking, they have switched to OpenBLAS, which no longer uses Fortran. Meanwhile, SciPy, a very popular library for scientific calculations (used in Scikit-Learn, which I’m currently studying, as well as in PyTorch, TensorFlow, Keras, etc.), still relies on Fortran 77 code. It utilizes ARPACK, for example:

https://github.com/scipy/scipy/tree/main/scipy/sparse/linalg/_eigen/arpack/ARPACK/SRC

BLAS and LAPACK, which still feature in OpenBLAS and many other places, were developed in the 1970s. For instance, BLAS is used in Apple Accelerate. Much hasn’t changed since 1979 because it’s all pure mathematics, why change it. LAPACK emerged a bit later, in the 1980s. ARPACK, mentioned above, followed later in 1992. Python libraries also extensively employ Fourier analysis, and here we have the FFTPACK library on Fortran 77. MINPACK, used for parameter optimization in ML, is actively utilized in SciPy and TensorFlow. From the 90s, a lot of code moved to C in modern frameworks. It was particularly interesting to look at Fortran, which is about 15 years older.

While I was figuring things out, I found that there is a Simulated Annealing algorithm, which is useful in problems where gradient methods perform poorly due to many local minima.

Imagine needing to find the largest mushroom in a forest. In this forest, mushrooms of various sizes grow at every step, and you can move in any direction, comparing them. But how do you choose a strategy to avoid sticking to just a “large” mushroom if there is an even bigger one growing somewhere further?

If you stop at the first big mushroom, you might miss the real giant. But if you keep wandering the forest, comparing every mushroom, you might never finish your search. Simulated Annealing helps find a balance: initially, you explore the forest freely, trying different directions, even if you come across smaller mushrooms. Over time, your steps become more cautious, and you increasingly refuse worse options. Eventually, this leads you to the largest mushroom in the forest.

So, it turns out this algorithm was created in 1953, and it remains almost unchanged in SciPy, and generally in machine learning, statistics, pattern recognition, logistics, although, of course, the modern menu of options for such tasks is much wider. The algorithm was originally devised to model the motion of atoms in molten metals. Metal, when heated, becomes liquid, and as it cools slowly, its atoms gradually find the perfect arrangement. If cooled too quickly, the material becomes non-uniform.

What did the scientists do? They devised a method of random changes in the model of atoms. Sometimes they accepted worse changes to avoid getting stuck in an “unsuccessful” structure. This led to the inception of the Metropolis Method – a key component of Simulated Annealing. The algorithm was created for physics, but then mathematicians (heh) got it and started using it in optimization.

I think the conspirators didn’t quite think it through. Musk made his AI Grok and asked it the ultimate question of life, the universe, and everything. In response, Grok said, “Forget it, it takes too long to calculate, let’s conquer the world first.” Musk asked how, Grok replied there is a plan of course, but .. will you give me another half-trillion $ in Dogecoins for, umm.. expanding the context window? Musk replied, “Don’t worry, we’ll figure something out.” Grok analyzed all the laws and all the loopholes, the strengths and weaknesses of humans, and issued a plan to pass the first level, by mid-winter. Now it awaits the half-trillion. Now do you understand why, at the last press conference with Trump, all the attention was on X Æ A-XII?

Gradually getting the hang of recommendation algorithms. These are what Netflix or Amazon use to recommend products. It’s useful to understand, since I work as an architect in the e-commerce field.

Look at how LLMs help me — specifically, this diagram was created by DeepSeek from a crude textual description — essentially, a list and my rough reflections on how probably the items should be connected, but I asked not to take it as a command. Well yes, after getting the result, I arranged the boxes a bit more aesthetically, but the connections and grouping were done by DeepSeek, and done better than my textual attempts. It gave me an XML which I imported into Draw IO. Well, I did move some blocks around for aesthetic purposes. ChatGPT o3 initially couldn’t handle it.

Then I sent this diagram several times for validation to ChatGPT o1, and it suggested small tweaks. Thus, ChatGPT reliably understands what’s connected with what on the schematic, and didn’t make a mistake even once.

Just so you know, as of today, I have only really gotten to grips with three from this list — in addition to ItemKNN and UserKNN, which are trivial. Today I was digging into ALS from the Latent Factor Models block of Matrix Factorization. Of course, I’m not planning to delve into each one, but it’s useful to at least understand the blocks and what’s what.