Today I sold a refrigerator. It has a story. The essence of it is that it’s not a refrigerator, although it looks like one. It’s a freezer. And it freezes on average to minus 18 degrees. I bought it second-hand, thinking it was a refrigerator. The buyer also came today thinking it was a refrigerator.
And here I realize that minus 18 degrees is not at all what I need.
Well, I am a Solution Architect. I didn’t want to dig into it, I just drove to Lowe’s and bought a simple blinker. It turns on and off according to schedule whatever is plugged into it. I stuck a radio thermometer inside (I had one) and adjusted the blinking frequency (20 minutes) so that the internal temperature was on average +4 degrees Celsius. The radio thermometer showed that the temperature fluctuations were very small – nominally plus or minus 0.5 degrees from +4, even less. And so it worked for me for some months until I realized that I just didn’t need it.
Sold it today with the adapter. It’s gone to the people.
We’ve all encountered it — the “Main Excel Spreadsheet Managing the Business.” The very one B2B companies use to calculate million dollar quotes. It has 12 tabs, 1000+ nested formulas, and zero documentation. For ten years, it had “quick fixes” slapped on and constants hidden away. It’s no longer just a file, but a living organism that no one fully understands except for the guy who quit years ago. That’s how puzzled I was. Moreover, there was uncertainty whether even half of the formulas were needed, or if they were vestiges of the past.
I was tasked with transferring this logic into code so that it was all computed by software. The Excel file seemed to have everything it needed, but in reality — it was a complicated black box. 1069 formulas.
The challenge was in how to translate a thousand interdependent formulas into clean code without losing any edge cases.
Here’s what I ended up doing.
Instead of rewriting everything from scratch at once with uncertain prospects of bug proliferation, I used a strategy of lazy computations and mocks.
I built a structure on Groovy that mimicked Excel’s behavior. Each computation (from a cell) I defined as a function that executed only when it was called. And the functions were a multidimensional dictionary.
I started from the end of the computation graph: from results to inputs. If a formula depended on something I hadn’t yet written, I “mocked” it in the code, simply substituting the value from the Excel sheet.
Bit by bit I replaced these mocks with real logic. Comparing the output of my code to the Excel at each step, I could clearly see where my logic diverged.
In other words, I moved from the result to the input data. At each step, it was clear which mocks needed to be turned into code, and I could compare version +1 with version -1 — the result had to match. As soon as all mocks were replaced with calls — the task was done.
The real “secret ingredient” was the dynamic nature of Groovy for creating a multidimensional map of functions. Instead of static variables, I used a deeply nested structure, where each “leaf” was a closure. This allowed access to any part of the table — be it an input parameter, a config constant, or a complex intermediate result — through a simple, unified syntax, and some components were dynamic.
x -> calculate[“Group”][“Subgroup”][it][“Someparameter”]() * conf[“someConstant”]()
}
}
Using dynamic keys and closures, I could iterate through product groups or data sets. Since these were dynamic functions, not stored values, the entire system worked like a living graph of dependencies.
Testing was possible right from the start of transferring the formulas. The charm was that you were kind of addressing a cell through syntax like calculate[“Totals”][“A”](), but in reality, you were launching an entire tree of calculations at that moment. And this was incredibly convenient for debugging.
In two weeks, the “Black Box” was transformed into a transparent, modular library with clear logic, which produced exactly the same result as the original table.
P.S. Of course, all the data in all the screenshots are thoroughly obfuscated, or rather, written from scratch for this text.
Oh, I just solved a really cool problem. It’s tricky to explain though. But I’ll try.
So, the client has 10 search websites. They all use one index but throw different queries at it. To what the user enters, a very long and complex query is added, generated by a module on Sitecore. It includes template and page IDs that need to be included or excluded. Ultimately, it’s impossible to understand what’s going on there. There could be ten opening brackets and some randomly closing ones, but it worked with Coveo. Reformatting helped, but not much.
And each site has its own version of this. Meanwhile, the same IDs appear periodically. I first tried to manually figure this out, but it was a nightmare. Nothing helped. There are also nested conditions. For example, “exclude this template” not globally, but only if that field equals one.
Here’s what I did:
I wrote a script that parses this textual “mess” into an abstract syntax tree (AST). This allowed to turn an unreadable string into a structured JSON object, where it’s clear: here’s AND, there’s OR, and here — a specific condition.
Then I turned these conditions into Boolean algebra formulas. Using the SymPy library, I “fed” these formulas to simplification algorithms. Mathematics itself eliminated duplicates, collapsed excessive nesting, and removed conditions that are logically absorbed by others. As a result, the “trees” became flat and understandable.
In the attachment — the original tree and the simplified one.
To be sure that I didn’t break anything during simplification, I wrote a test generator. It takes the simplified logic, puts it back into a working curl, and checks whether the number of found documents (totalCount) matches the original request. The numbers matched — meaning, the logic is preserved 100%.
Having simplified and standardized structures for each site in hand, I built a comparison matrix. The script analyzed them and highlighted Common Core — conditions that are guaranteed to be required (or prohibited) on all sites without exception, and Specifics — unique “tails” that distinguish one site from another.
In the attached screenshot: REQ means that the condition is guaranteed to be met for any document that goes through this request. NOT — definitely not met. OPT — the condition is present in the request, but it’s not strict by itself. It only works in conjunction with something else. “.” — the condition is not mentioned in the request at all.
For 3 sites it responds instantly, for 10 it takes about 30 minutes.
And of course, all data in all screenshots are thoroughly obfuscated.
I’m checking out what open data we have in our county to play with data analysis over the weekend, and discovered, for instance, an open database of all 1.5 million trees in the county. The screenshot shows just a tiny part around my house.
While figuring out neural networks, I decided to come up with a game-related task for myself. What if I find some ready-made games, and train a neural net to predict moves based on the board situation. Said and done. Of course, generating code is faster with LLM, but I wrote the detailed assignment myself and designed the architecture on my own. In 40 minutes (!) from the idea to the result, I already had a working solution that, at least in the first half of the game, does not mess up too much.
In the screenshot is CuteChess – it works with any chess engine, and in my case, it’s a simple Python script. The script takes the board situation and feeds it to the model. It selects the top 5 moves, and only these top 5 are analyzed deeply for several moves ahead and assesses the position. That is, the neural network suggests possible moves based on the analysis of 20,000 games (534,453 positions). From the results, the best is chosen. It uses the minimax algorithm for this, if that means anything to anyone (it didn’t to me, so Gemini here helped me)
How the model is trained. On the lichess website, you can download games, there are hundreds of gigabytes. I took a file with 800,000 played games from the year 2014. From these 800,000, I select 20,000, specifically looking with a script for games where the result is not a draw (1-0 or 0-1). Next, I calculate the difference (Winner_Rating minus Loser_Rating). It’s not the best metric, but it’s better than nothing. The bigger this difference, the more “confident” the win should be (the strong punish the weak). Thus, I get 20,000 such games.
“Ignoring the moves of the weak” (to avoid teaching the model bad play) is implemented during the training stage of the model. Essentially, the logic is: “If it’s White’s turn now, and White won this game — we learn. If it’s Black’s turn now, and Black lost — we skip and don’t teach the net this move.”.
The neural network is trained in batches of 128 positions at a time. The network receives a board position as input and outputs 4096 — the probability assessment for each possible move.
Selecting games takes about 5 minutes. Training the model on my computer takes about 10 minutes for 20,000 games. You could leave it to train on 100K or a million, and it would definitely be better. No need anymore – I figured it out 🙂
I am currently experimenting with training simple neural networks – primarily to automate the existing toolkit, and some things just seem like magic.
There is a database of 32,000 names. There is a neural network filled with random numbers. I start training, with only this list of names as input. The first layer of the neural network is embeddings, and I set the number of dimensions to 2 for easy visualization. And after 200,000 iterations of training, the system clearly separates vowels from consonants, and for some reason, places the letter “q” slightly apart from other consonants. It seems that this is because the letter ‘q’ almost exclusively predicts the letter ‘u’ (Queen, Quincy, Quentin).
It also very reliably separates vowels and consonants in Russian names. In Russian names, the letters b and l are somewhat away from the other consonants, as are the soft and hard signs (well, that’s understandable).
I wonder how it works. If trained on a normal corpus of texts, the difference would be very clear. Why are vowels separated from consonants? Apparently, from the network’s mathematical perspective, ‘a’ and ‘o’ serve the same function: they “trigger” the prediction of the consonant following them, so the alternation of vowels and consonants is to blame. But damn, it’s interesting 🙂
And since the model can predict the next letters, you might try running it on Russian. On a model with 30-dimensional embeddings, it invents names like: Byaketta, Afsena, Erakey, Zasbat, Daraya, Gaiomahad, Rain, Razhul, Gzhatsiy, Reben, Vureb, Durodira, Turuzhul, Regravgava, Razsan, Gabila, Avganzh, Raksi, Khalebkokhorta, Rather. The model – for those who understand – is this: input of 6×33 characters (because we take up to 6 characters of context), encoded into embeddings of 60, goes to a layer of 100 neurons, and from there back to 33 characters. Some nonsense, but at least it’s clear how it all works at all levels.
An interesting study caught my eye, dating back to 2009. According to it, the modern human indeed reads significantly more than in the past, although the format of this reading has changed. The study suggests that in 2008, an average American consumed about 100,000 words a day (approximately a quarter of “War and Peace”) – this is an approximate number of words that passed through consciousness per day (via ears or eyes), calculated based on activity chronometry. This is 140% more than in 1980.
Therefore, contrary to the myth about the degradation of reading, at least in 2008, we processed 2.4 times more textual information than our parents’ generation. Moreover, the study only considered information consumed outside of work (at home, in transit, during leisure).
The structure of reading – if in 1960, 26% of words came from paper, by 2008 this share had fallen to 9%. However, digital media (internet, email, social networks) not only compensated for this decline but also tripled the total reading time. The reason — the internet, as it is predominantly a textual environment (web surfing, email).
But it’s interesting that although the Internet accounts for 25% of consumed words, it only makes up for 2% of bytes (since video on the internet in 2008 was of low quality). Thus, they estimated the information flow from different channels and converted it into bytes 🙂 Radio accounted for 19% of the time but only generated 0.3% of bytes (as audio requires less data). Voice communication (telephone) — accounted for only 5% of words and a negligible share of bytes, but it was the only fully interactive channel before the internet era. TV remained the main source of information by time in 2008 (41% of all hours) and quantity of words (45%), however, in terms of data volume (bytes), television was only second (35%), behind computer games.
Now about games, quite interesting. The main finding from the report: Games generated (or did in 2008) 55% of all “bytes” consumed by households. Meanwhile, they only accounted for 8% of user time. This is quite a controversial topic in their report.
Those 100,500 words — that’s an assessment of actual words that a person either read or heard. This is not a metaphorical “equivalent,” but an attempt to calculate the verbal information precisely. They took the consumption time of each media and multiplied it by the average word inflow rate for that channel. Reading (books, newspapers, internet texts): 240 words per minute. Email and web surfing – 240 words per minute. Television (dialogues in shows/movies): 153 words per minute. Radio: 80 words per minute (less because of many pauses and music). Music: 41 words per minute (song lyrics).
I kept seeing ads for an AI language tutor that I ignored, and the system forgot about me for a while before coming back with a noticeably older tutor.
But really, how soon will video advertising become personalized for us? Where in the same ad, New Yorkers will see their city, black people will see black people, in the morning the main character will be drinking coffee, and a car with the logo of their alma mater will flicker in the background?
In Soviet times, there was a great school of animation that led the world for many decades. If you search on YouTube for “Vovka in the Land of Far Far Away”, it almost exclusively shows restorations 🤮, and at the same time, it shows the same disgusting restorations of heaps of other cartoons, all made in the same style (vectorization, black outlines). If you go to Wikipedia, it will display a screenshot from the restoration, not from the original 1965 cartoon. The original can be found, for example, by searching “vovka in the land of far far away madina gazieva”, but searching “vovka in the land of far far away soyuzmultfilm 1965” shows nothing at all.
They really broke the internet.
P.S. By the way, “two of a kind, fulfilling wishes,” and “good enough” resonate very much with today’s ChatGPT 😉
Hi, I'm Rauf Aliev. 