Tag: science

Rediscovering the 1986 “Chemical Trainer”: A Pioneer in Interactive Learning | November 23 2025, 15:55

At my home in Kolomna, I have a book called “Chemical Trainer” from 1986. I have never seen anything like it before or since.

The material of each of the 54 programs is divided into many small, very short sections, or categories. At the end of each category, one or more questions are posed. This is done to check whether the content of the category is truly understood. For each answer, there is a place in the book to jump to in order to see if the answer is correct. If the answer is wrong, it describes why and asks a new question. If correct — you move further in this quest.

These Germans in 1986 created an interactive textbook even before it became fashionable.

Exploring the Fascinating Properties of Glass | November 21 2025, 23:58

I got carried away with the topic of glass and learned so many interesting things, so I’m sharing. It all started when I read about the supercritical state of matter – it turns out that the line separating liquid and gaseous states on a pressure and temperature graph at some point breaks off, and beyond that lies a state of matter that is neither here nor there. I started reading about states (phases) of matter and stumbled upon the fact that glass is essentially a state between liquid and solid. It flows, just very slowly. This myth is popular thanks to observations of medieval windows, where the glass is often thicker at the bottom, which was attributed to “flowing” under the influence of gravity, and it was even mentioned in school textbooks. In reality, glass is an amorphous solid with extremely high viscosity at room temperature, and it does not flow noticeably even over billions of years; the uneven thickness of old glass panes is explained by production technologies, when the thicker edge was installed at the bottom for stability.

I delved into the topic of glass further. It turned out that the reason why glass can be transparent is rooted in quantum mechanics, specifically in the electronic structure of the material, not because of the density of particles. The essence is that for an electron to absorb a photon, it must transition from one energy level to another, but in silicon dioxide, the width of the band gap is so large that the energy of visible light photons is physically insufficient to make this “jump.” As a result, light simply cannot interact with the electrons and goes straight through the material, while higher-energy ultraviolet radiation can overcome this barrier and is thus absorbed by glass.

It also turned out that melted glass conducts electricity. Moreover, the mechanism of conductivity fundamentally differs from how metals conduct electricity. In a copper wire, current is a flow of free electrons. In cold glass (an insulator), electrons are tightly bound, and ions are locked in the solid lattice. But when you heat glass to the molten state (usually above 1000 degrees for silicates), thermal energy breaks the rigid bonds of the lattice, and glass becomes a liquid, with ions gaining freedom of movement. The current in molten glass is the physical movement of charged atoms (ionic conductivity), not just “flowing” electrons.

The green tint you see on the edge of regular glass (as seen in the attached picture) turns out to be caused by iron ions, present as impurities (~0.1%). Sand is a natural material, and removing all the iron from it is difficult and costly. Low-iron glass, which has tens of times fewer iron ions, is used in solar panels, not just because it is more transparent. Iron greedily absorbs the infrared spectrum (thermal energy), reducing the efficiency of the panel. By removing iron, we allow maximum energy to reach the silicon cells.

And finally, the most “mind-blowing” (literally). There are these things called “Prince Rupert’s drops.” If you drop molten glass into icy water, the outer shell of the drop cools and hardens instantly, while the inner part remains liquid. As it cools, the core tries to contract, but the hardened shell doesn’t allow it. As a result, the inside of the drop preserves colossal mechanical stress (up to 700 MPa).

The physics of this process creates a paradox: the “head” of such a drop can withstand being struck by a hammer because the compression of the surface makes it incredibly strong (the same principle is used in tempered glass for smartphones). But just nick the thin tail, and the balance of forces is disrupted, and a wave of destruction moves through the drop at the speed of a bullet (about 1.5 km/s), turning it into glass dust right in your hands.

There’s also something in physics called “metallic glasses” (amorphous metals). If you cool the molten metal at a rate of a million degrees per second, atoms do not have time to arrange into a crystalline lattice and freeze in chaos. Such “glassy metal” possesses unique magnetic permeability and is stronger than titanium, because it lacks crystal lattice defects, which are usually the points of destruction. So glass is a much broader concept than just transparent substance in our windows 🙂

The only example of an object made from this material, amorphous metal, that I’ve encountered is, believe it or not, the iPhone clip.

By the way, that same amorphous structure of glass, which I mentioned earlier, gives it an unexpected advantage — supernatural sharpness. If you take a scalpel made of the best surgical steel and look at it under an electron microscope, its edge will look like a jagged saw. This is inevitable: steel is made up of crystalline grains, and it’s impossible to sharpen it any smoother than the grain size allows.

But obsidian (volcanic glass) when fractured provides an edge only about 3 nanometers thick (about 1/30000 the thickness of a human hair). There’s no magic here, just that glass lacks a crystalline lattice, which would otherwise prevent achieving a perfectly smooth fracture down to the molecular level. That’s why obsidian scalpels are still used in the most complex eye surgeries — the cut is so clean that tissue cells are minimally traumatized, and healing occurs faster.

And one more powerful engineering case — vitrification (glassification). Mankind has chosen glass as the most reliable “safe” for nuclear waste. Liquid radioactive waste is mixed with special additives, melted, and cooled into blocks. The trick is that dangerous isotopes are not just poured inside, they are chemically embedded into the atomic grid of the glass. Glass is chemically inert, it doesn’t rust like metal or decompose for thousands of years. This is perhaps the only material that engineers trust to store hazardous substances on a geological time scale. Yes, it takes about a million years for a discarded bottle to decompose.

And finally. Digging into history, it turns out that the Romans were engaged in nanotechnology 1600 years before we even invented the word. In the British Museum stands the “Lycurgus Cup” (4th century AD). If you look at it under normal lighting, it’s greenish and opaque. But if you place a light source inside the cup, the glass flashes bright rubin red.

Until the 1990s, scientists could not understand how this was achieved. An electron microscope showed: Roman craftsmen added gold and silver, ground to nanoparticles about 50 nanometers in size (about 1000-1800 times thinner than a hair). This size of particles triggers a quantum effect known as surface plasmon resonance: electrons in the metal begin to oscillate such that they absorb some wavelengths of light and let others pass depending on the angle of incidence. The funniest thing is that the Romans did this empirically, “by eye,” and we’ve only just learned to replicate this consciously in photonics. It’s crazy to think you could handle 50 nm gold dust by eye. This moment required additional googling.

It’s unlikely the Romans mechanically crushed the metal to 50 nanometers — they had no such mills.

More likely, they added gold and silver in the form of salts or foil to the molten glass mass. The nanoparticles formed not by crushing, but by crystallization and sedimentation from the melt under very precise temperature conditions (“glass prescription”). This is even more complex chemistry than simple grinding.

The most astonishing thing is not that they did it, but that the ratio of gold to silver was maintained perfectly. Changing the concentration of gold by just 1% would alter the color to something other than pure ruby red. This indicates that the craftsmen mastered the technology incredibly accurately, although they likely did not understand the mechanism. And that they had a heck of a lot of time for all kinds of nonsense;) probably many generations dedicated their lives to experimenting. Because it’s hard to see why all this was necessary.

There’s a beautiful hypothesis (unproven, but popular) that the cup could have been used as a detector. If you pour a different liquid into it (for example, alcohol with impurities or poison), the refractive index changes, and the color of the “flash” might vary.

Data Science: The Modern Alchemy of the 21st Century | November 16 2025, 04:02

A cryptic post today. While writing a book on RecSys, I caught myself thinking that modern data science is essentially the alchemy of the 21st century. Half of the “best practices” in algorithms lack a solid mathematical framework. It’s a set of heuristics that “just work”. Much like in the 17th century where they mixed everything indiscriminately, it happens now, and if something works better, everyone else starts doing the same. There’s just no answer to the question “why”.

Take, for example, the NCF/NeuMF (Neural Collaborative Filtering) algorithm. The logic goes like this. Say, there are a million movie ratings by users. And 100 million ratings by users yet given – users can’t watch every movie in the world. But out of these 100 million, you need to choose candidates for advertising for a particular user. The algorithm, of course, has a training phase, where weights are calculated, and a prediction stage, where these weights are used on the incoming data.

(What the algorithm does. Essentially, it’s an ensemble of three sub-algorithms, two of which generate their own conclusions, and then their decisions go to a new neural network, the third algorithm, which provides the final recommendation. Smartly, it’s a hybrid of GMF (matrix factorization) and MLP (Multi-Layer Perceptron). The first of these two is based on matrix decomposition, and the second represents a neural network with multiple layers. Weights are adjusted on training data.)

For one positive example, it takes 4 negative ones. Why four? Just because it’s “not too many and not too few”. Would 8 be better? Unknown, but it would definitely take longer to learn.

Why are embedding dimensions 32? or 64? There’s no formula. It’s the “golden mean” between a “dumb” model (few k) and an “overtrained” (many k).

Now about the neural network. Why is the MLP block built as a “tower” (64 -> 32 -> 16)? Why not (50 -> 25 -> 10)? Why ReLU between them (and not tanh for example)? Pure empiricism. The number of layers in the tower is also adjusted.

Why do GMF and MLP parts have different embeddings at the input? Because the authors of the paper tried it, and it “worked out better”. No mathematical proof. Why do they go to the final layer with equal weights? Because they just do.

Why are the outputs of the two paths “concatenated” (concat), and not added or multiplied? “Experience showed that this way the result is more accurate.”

And so it is with everything, up to the choice of optimizer Adam or the “magical” learning_rate=0.001, although at least these have some mathematical basis.

That is, at least a dozen parameters of one algorithm are empirically chosen, with no clear confidence that they are independent of each other. But many of them depend on the dataset, but no one knows how 😉

In general, alchemy.

Metchnikoff: Beyond Science and Survival | November 13 2025, 04:53

I was reading Metchnikoff’s biography (don’t ask why I ended up there) and thought about how much can fit into one life. He wasn’t just a scientist, but rather like a saga:

His elder brother Ivan was the prototype for Leo Tolstoy’s “The Death of Ivan Ilyich.” Another brother, Lev, was a prominent anarchist, sociologist and fought in Italy alongside Garibaldi. Metchnikoff himself tried to end his life twice: the first time after the death of his first wife (who, sick with tuberculosis, was carried to the church on a chair). He took morphine but survived. The second time was when his second wife Olga fell critically ill with typhus. He deliberately inoculated himself with relapsing fever. Fortunately, both survived. However, the Grim Reaper with his scythe only came after his third consecutive heart attack.

The dude graduated from university at 19 as an external student. I.M. Sechenov himself recommended him for a professorship. But Metchnikoff was “blackballed” (rejected) by one vote. In protest, Sechenov resigned along with him.

He founded the first bacteriological station in the country at that time in Odessa. But due to an employee mistake (they spoiled the anthrax vaccine) an entire flock of sheep died. After this scandal, he left Russia. The station — on Leo Tolstoy Street.

In Paris, he was immediately taken under the wing of Louis Pasteur (the father of pasteurized milk), who supported his theory and gave him a lab in his institute. There, Metchnikoff worked for 28 years, becoming the deputy director.

While studying cholera at the Pasteur Institute, Metchnikoff proposed a theory that not everyone who comes into contact with the pathogen gets sick. He suggested that it’s all about… (of course) the gut flora. To prove it, he deliberately drank a culture with cholera vibrios. Nothing happened (it would have surely happened to you, Metchnikoff thought)

In the end, he received the Nobel Prize for the discovery of phagocytosis (cellular immunity). He is also “the father of gerontology” — Metchnikoff was the one who proposed the theory that to achieve longevity, one must combat bad bacteria in the gut with probiotics. Now, they say, gerontologists around the world drink sour milk on May 15th remembering Metchnikoff.

He died in Paris, and his ashes are kept in the library of the Pasteur Institute.

Also, in the English Wikipedia he’s Élie Metchnikoff. Not easy to guess.

In the photo, Metchnikoff and Leo Tolstoy are discussing immunology.

Exploring Recommender Algorithms Through Interactive Visualizations and Sandbox Simulations | November 11 2025, 05:23

I’ve launched an electronic open source application for my book Recommender Algorithms! It’s a “sandbox” where you can “run” various recommendation algorithms with different settings, and view specific visualizations for each algorithm that help understand how it works. For instance, for algorithms like ItemKNN, SLIM, or EASE, a key visualization is a heatmap of the learned similarity matrix (item-item similarity matrix). This allows you to see which pairs of items the model considers “similar” (or “influencing” each other). For SLIM, for example, a useful “Sparsity Plot” shows that the similarity matrix indeed turned out to be sparse. For associative rule algorithms (Apriori, FP-Growth, Eclat) the visualization is not a graph, but interactive tables with found “Frequent Itemsets” and generated “Association Rules,” which can be filtered and sorted.

Additionally, there is a parametric mechanism for creating a “game dataset” — Dataset Wizard. It works like this – there are template datasets that describe items through characteristics. For example, recipes through flavors. Or movies through genres. The system generates random users with a random set of characteristics from the same set — and there are many sliders to make this distribution more contrasted or complex. Next, a matrix of user ratings of items is created – conditionally, if the characteristics of the user and the item match, then the rating will be higher because “tastes match”; conversely, if they differ, then the rating will be lower. Here too, sliders add noise and scarcity – randomly removing part of the matrix. The characteristics of products and users are not fed into the recommendation algorithm; they are hidden, but they are used to visualize the results.

The third component of the application is the tuning of hyperparameters. Essentially, it’s an auto-configurator for a specific dataset. An iterative approach is used, which is much more efficient than a full search (Grid Search) or random search (Random Search). In short, the system analyzes the history of past runs (trials) and builds a probability “map” (surrogate model) of which parameters will likely yield the best result. Then, it uses this map to smartly choose the next combination to test. This method is called Sequential Model-Based Optimization (SMBO).

The code is open source and will be further supplemented with new algorithms and new visualizations.

Link to the code in the comments.

Link to the site where the code is deployed and where you can check out the application is also in the comments.

Samuel Morse: From Painter to Telegraph Pioneer | October 28 2025, 15:00

At the “Rzhipopisi” exhibition, a painting titled “Paris through the Eyes of Samuel Morse” was showcased. Essentially, dots and dashes—it sparked the idea for this post. Few know that Samuel Morse was actually an artist, and quite a decent one—check out a couple of his paintings attached to this post. But he was only “decent” by our standards—surrounded by many equally skilled artists, he considered himself a failure in this realm and devoted the second half of his life, 35 years, solely to the telegraph. (By the way, Hitler was also an artist, amateurishly decent, but more mediocre compared to Morse amidst his contemporaries, yet he ventured into politics). In the attached photos, there’s a painting with paintings. Its actual size is about two meters and among the paintings hanging there is even the Mona Lisa (La Joconde) by Leonardo da Vinci, which wasn’t valued back then as it is now. It mainly became famous after it was stolen from the Louvre, and then fervently searched for and found by the entire world.

By the way, Morse Code was not invented by Morse, but by Alfred Vail, his colleague—a fact Morse later repeatedly denied (while also attributing the invention of the telegraph itself to himself). In 1848, the Vail/Morse code was refined by the German Friedrich Gerke. The code, improved by Gerke, was used until new technologies came along.

(By the way, I don’t understand why it’s Morse and not Morz. He was American, and nobody ever called him Morse.)

Indeed, among people who were artists, about whom everyone has forgotten that they were artists because they remembered something else, it is worth mentioning besides Hitler, also Winston Churchill and George W. Bush Jr.

Unveiling “Recommender Algorithms”: A Comprehensive Guide on Recommendation Systems | October 25 2025, 17:36

I finally released a book on #RecSys! It’s called Recommender Algorithms, where I’ve compiled over 50 recommendation algorithms with detailed mathematical derivations, thorough explanations, and code examples.

https://www.testmysearch.com/books/recommender-algorithms.html

It all started early this spring in Germany, when I attended an ACM conference and sketched out the first structure of the book while analyzing the talks from the RecSys track. And now, just six months later, it has come to life.

Why did I write it? Because neither online nor in print is there a single, accessible resource that deeply explores recommendation algorithms of various types and purposes. There are articles focused on small subsets, but collecting and systematizing approaches—from foundational methods to the very latest—seems to have never been done before. I don’t know if I succeeded, but I’d love to hear your feedback.

Please like & share!

P.S. Click at READ SAMPLE to see the first 40 pages. The table of contents is there as well.

https://www.testmysearch.com/books/recommender-algorithms.html

https://www.testmysearch.com/books/recommender-algorithms.html

AI Salesbots at Your Door: The Future of Autonomous Presentations | October 16 2025, 15:47

I’m telling the manager now, why do we need to present our AI solution, it’s AI, let it present itself. I imagine that in the near future, bots will be knocking on doors to sell themselves (and maybe not just themselves), while the door will have built-in bot protection.