Donenfeld: Random number generator enhancements for Linux 5.17 and 5.18

- Commit messages describe justification for _changes_. That is, a rationale for going from one state to another state. This discussion generally includes some research on the old state in order to motivate the new state, in addition to information about the new state.

- random.c's code comments and section headers now have lots of up to date information. These describe the present state and are useful as documentation, especially to people hacking on the code, since it's beside the code.

What we don't currently have but something I am working toward is a detached description of the RNG and motivations for various design decisions. This will be useful to researchers and such trying to get a good high level description of how the thing works and motivations for its various components. I expect for this aspect of the documentation to congeal as the RNG's design does as well. But hey, already what's happened in 5.17/5.18 is a pretty big improvement on the status quo.

git log --grep works pretty well.

On the contrary - I find it one of the easiest places to look. I use use all of `git log --grep`, the GitHub search bar, `git gui blame`, `git log -S` (pickaxe) and the VSCode git lens depending on what I'm looking for and what environment I'm using.

The beauty of it is that it's context, context, context.

* Commits (including but not limited to the commit message) provide context for a changed line. I try to provide as many references in my commit messages as I can. This includes references to other commits, or links to bug-trackers or documentation or stack-overflow posts, etc.
* Commits also exist in their own historical context - both in terms of other commits on the branch, what was happening on other branches around the same time, and how and when the commit was merged. The beauty of this is that it happens automatically.
* This context is immediately available in your editor when looking at the code too. With git blame and associated editor integration you can see how the code you're looking at came to be like this. The "why" of some code is immediately accessible. You might argue that this information should be in the comments, and in some cases this true, but you can be a lot more verbose in a git commit message than in a code comment, and with blame *every line* has some message associated with it, with additional context a few clicks away.

> Ideally this kind of explanation should be in nicely structured text files.

I think separate documentation works best for high-level design type documentation that's unlikely to change quickly, and user facing documentation. Less so when things change quickly, or need to be cross-referenced with code.

The beauty of commit messages is that the message is attached to the code, and additional context is captured automatically. With separate documentation you need to refer to the relevant parts of code (and keep those references up to date). You also need to manually include a bunch of context so the future people can understand the documentation - much of which would be included automatically by git. It's also harder to find the relevant documentation from the code, while from any given line the commit message is just a click away. Documentation gets out of date - so do commit messages, but at least with commit messages you can know when it wasn't out of date and you can see exactly the code that it applied to.

Also: if you're writing documentation why not include it in the commit message too? Copy and paste is easy and there's no harm to the duplication. Sometimes I feel like developers are so used to applying DRY to their code that they get carried away and don't ever want to press Ctrl-C, Ctrl-V.

I can understand how people feel differently. A lot of the value of these tools only come if developers take care to make their commits atomic and provide useful information in their commit messages. Worse: you can individually start applying this discipline to a project but you won't start seeing the benefits of doing so for at least several months - it's only really helpful once you've forgotten the context in which the code you're looking at was written.

No, it does not.

> We should try to capture as much as possible of our meaning in that code, where a maintenance programmer is obliged to change it if they change the intended meaning later, whereas the comments can become out-of-sync with the code either through oversight or misunderstanding.

Yes, they can. However, they are hard to ignore if without a comment one is unable to understand the code.

> Modern optimising compilers can assist greatly here by allowing us to more often express what we actually meant,

They are useless.

> this is one of the opportunities for Rust once that's an option.

It is completely useless.

> C doesn't even have iterators, you can't express "are all the doodads in this bag of doodads sparkly?" except by explicitly enumerating them

Iterators don't help.

Let me explain what kinds of detailed comments are useful (picked at random from a codebase):

- «Calling this external API simultaneously using the same API key causes it to lock up, so there is a mutex»
- «Detaching a disk from a VM before Linux has booted causes a desynchronization of disk names inside the VM and in the libvirt, so wait for an r/w activity on the disk as a telltale of mounted root filesystem before detaching it»
- «This certificate is used in a closed-loop system, so CA/B requirements do not apply to it»
- «Here's the reverse-engineered description of undocumented IPC»
- «mount(2) contains a lot of historical baggage, and that baggage, unfortunately, got leaked to the external system interface, so the following code mimics the mount(8)»

These are facts about other systems.

Sorry, that's one of the Big Lies of programming. It didn't work with COBOL, it doesn't work with C, and it certainly won't work even with the Great New Pie-in-the-Sky language. I've heard variations on this statement for over 45 years and it's never been true. As long as we have computers that operate step by step rather than intuiting what we mean, we'll need comments in and on the code.

The code cannot answer two critical questions about itself...

A) WHY the code does its thing.

B) What was intended, as contrasted with what the code actually does.

Without knowing A it's difficult to change the code with confidence.

Without knowing B it's difficult to know what is a defect and what is intentional behaviour.

In a perfect world, sure. But humans notoriously bring their own degree of understanding and context to the use of any language, even languages intended for programming; and no nominal evaluation of their qualifications will always be effective in in assuring they're prepared to deal with what to them is obscure. Until all programmers are replaced by AI's coding everything else, we're stuck with supporting functional behavior despite imperfection. Maybe even then, if you're paranoid. :-)

An (old-timer) example:

while (*dst++ = *src++)
;

For someone familiar with C idiom (esp. if they learned from e.g. K&R), no big deal. Otherwise, although that's correct and concise for copying null-terminated strings (or 0 terminated arrays of any integer-like variable, so e.g. *src and *dist could refer to wchar_t rather than char) to a destination presumed to have sufficient space, it's got a LOT going on in a short space. It causes brain bleeds for some trying to learn C (pointers being a common stumbling block, and while this is well-defined, pre-increment combinations may not be; someone might be tempted to generalize wrongly). Using subscripts may be more readable, but I suspect this is among the more efficient ways to do it. It could perhaps be made more readable as

while ( (*dst++ = *src++) != '\0')
;

and perhaps still generate the same code if the compiler is smart enough, give or take the additional constant '\0' stored in the program.

Things like that are more or less idioms that one simply has to learn and recognize, even if one can also aspire to analyze them in painful detail. Even

/* idiom: see K&R */

would be some help there.

Moreover, some languages or projects have conventions for embedding documentation in comments, so it's all (ideally) kept current together. While I don't know of a specific example, that might include not only user or API documentation, but maintainer documentation too, with perhaps either separately retrievable from the source file. That might or might not be over and above inline comments.

OTOH, comments like

/* OMG, deadline is f-ing killing me! */

probably don't contribute much, give or take historical context and a possible warning of quality issues. And your future AI replacement will probably waste cycles dealing with humorous comments. :-)

That's exactly what I do. Also, if the MR/PR consists of a single commit, both Github and Gitlab will copy its message to the MR/PR description. It's very convenient, as it saves me from doing the copying myself :)