Systemd again (or how to obliterate your system)
Ok, I have been silent about systemd and its being forced onto us in Debian like force-feeding Foie gras gooses. I have complained about systemd a few times (here and here), but what I read today really made me loose my last drips of trust I had in this monster-piece of software.
If you are up for some really surprising read about the main figure behind systemd, enjoy this github issue. It’s about a bug that simply does the equivalent of rm -rf / in some cases. The OP gave clear indications, the bug was fixes immediately, but then a comment from the God Poettering himself appeared that made the case drip over:
I am not sure I’d consider this much of a problem. Yeah, it’s a UNIX pitfall, but “rm -rf /foo/.*” will work the exact same way, no?Lennart Poettering, systemd issue 5644
Well, no, a total of 1min would have shown him that this is not the case. But we trust this guy the whole management of the init process, servers, logs (and soon our toilet and fridge management, X, DNS, whatever you ask for).
There are two issues here: One is that such a bug is lurking in systemd since probably years. The reason is simple – we pay with these kinds of bugs for the incredible complexity increase of the init process which takes over too much services. Referring back to the Turing Award lecture given by Hoare, we see that systemd took the later path:
I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. Antony Hoare, Turing Award Lecture 1980
The other issue is how systemd developers deal with bug reports. I have reported several cases here, this is just another one: Close the issue for comments, shut up, put it under the carpet.
(Image credit: The musings of an Indian Faust)
> we pay with these kinds of bugs for the incredible complexity increase of the init process which takes over too much services.
The bug in question has nothing to do with the init process. It is in another component provided by systemd (the project — not systemd, the init process).
I agree Lennart’s reaction was horrible. However, I have to say I would find your complaints about systemd more compelling if you would point out an alternative. I for sure am very happy that I now know how to configure the clock, or the locale, or other things standardized by systemd — no matter which distro I happen to be on. Pre-systemd, this was a per-distro mess.
Fully agreed!
I don’t think paying the prize of being locked into a single system is worth having the convenience items of knowing how to configure the clock etc. Alternatives there are a lot, and even Debian has seen forks to allow for diversity. Unfortunately, already before but even more after the TC decision, the whole infrastructure is moving to systemd not supporting other init systems anymore, simply due to the reason that systemd takes over more and more responsibility. As Debian Developer I don’t plan to leave Debian, so I have to live with these changes, but I am less than happy.
Sysv was working fine for users, perhaps not for devs. OpenRC runs on my desktop, upstart was stable…
I for sure am very happy that I now know how to configure the clock, or the locale, or other things standardized by systemd — no matter which distro I happen to be on. Pre-systemd, this was a per-distro mess.
So, in the ideal world – for example – all cars should be exactly the same? Have the same engine, the same chassis, the same dashboard – work all the same? The only difference would be the logo? That would be a very bad thing. Literally, you would not have any options, but one. The same applies for Linux distros: diversity is good, for there is no universally good solution, just different solutions with different advantages and disadvantages.
I think you should get used to it, that different people thinks different, works different and so their stuff. It’s a much-much better way to learn different approaches than standardize a very-very wrong approach and systemd is the worst imaginable approach. systemd is an abomination and should be eviscerated from the Linux world.
In other words, systemd sucks.
An alternative? For example – use an init system THAT DOES NOT FUCKING REMOVE /
Why are you advocating for systemd by the way, Ralf ? Employed by Red Hat?
Very interesting article with what I can agree!
I do also agree that systemd unifies some commands over different distros. And it makes some things easier.
But I do not like the way that one program does so many different things and in such a complex way. I would rather prefer a per distro setting but with maintainers listening to the users and not strictly following their own ideas! And I do not talk about disadvantages like a shutdown takes much longer now than it was with other init-systems etc. My distro (Arch) as well as Debian (professional use) both use this init-system for now.
If a distro has a certain level of development I would expect an easy to find/read wiki with the default settings. So this is just no problem for me.
I think it’s about owning your bugs. These guys goes directly to “working as intended” and closing the bug, instead of saying “Shit, you’re right. That’s a really stupid mistake. We’re fixing it right away.”
..*exactly* the very problem with my fellow countryman Mr Poettering. This style of behaviour does not convey trustworthiness. Why does he act like he possesses papal infallibility? This is hybris waiting to come crashing down, just as pointed out here and illustrated in the article’s sheeple cartoon.
“The other issue is how systemd developers deal with bug reports. I have reported several cases here, this is just another one: Close the issue for comments, shut up, put it under the carpet.”
Just commenting on this case. The problem has been resolved, Poettering learned something (hopefully), what constructive things can come from leaving the issue open now? All I can see happening (now that it has been dragged out into public view) is a completely off-topic flame fest that helps nobody.
YMMV, but someone responsible for a critical software such as the initializer trying to blame UNIX to justify a catastrophic bug in widely released software as “[not] much of a problem” would get immediately fired if employed in my company, no matter how useful the rest of his software is.
It is not “off-topic” – it deals exactly with situations where systemd will do something that is a BUG.
How can this possibly be off-topic?
And what is “dragged into public view” – do you prefer people to not know about this behaviour?
Guys, don’t be such oversensitive pussies. Try sending such bug reports to Linus and you’ll get a much worse response.
Try pull requests… and better make it good ones.
Besides, every time I read complaints like these, I see systemd being, either, minimised or unknown by the user. What about all the awesome functionality? Encapsulation and resource management (cgroups), watchdogs, sockets on-demand, timed tasks (cron), super easy service management, etc. Really… there is no comparison.
Well, the question is, does all this super-trouper functionality belong there? And is it developed by the right team? One team to rule them all, and bring them into darkness…
Sorry Norbert, than simple don’t use it. I find these flamefest very annoying. Yes Lennart could have said, “ok, thanks for the input, I learned something new”, but that he closed the ticket is 100% fine. You are not a developer of systemd, and if you think that you can build a better team/better project for the task, than just do it. Don’t you think that would be more productive?
And yes I like these features, and yes much of these can be done using the old systemV scripts (not all of them!). But at no time this was easier, and cleaner – and yes that’s a big point for me. And sure this software has bugs, every software has bugs. So cool, somebody found a bug, it was fixed, Lennart learned something new. Everybody wins!
Question our leader and we will stand up and defend him justly!?
Yes, Mr Torvalds uses harsh tone and management style as well. But seldomly does he display such lack of foresight and subsequent reasonable humility as does Poettering’s reaction to systemd bug #5644. And this ain’t not the bloody first time.
There are very real dangers here — billions of Linux devices are to be deployed in the next decade, most of them running systemd. Security through self-perceived invincibility is not a viable FMEA strategy.
Guy, don’t be such an ignorant chump. Calling other folks names doesn’t make them more sympathetic to your position any more than me calling you an ignorant chump makes you any more sympathetic to mine. Of course, I don’t give a fraction of a fiddler’s foot funk what you think, because you are, apparently, an ignorant chump. My comment, rather, is intended for those whom you’ve called “pussies”, because you are too ignorant, and too much the chump, to respect the pussy.
Neither cgroups nor cron are part of systemd. cgroups are part of the kernel, and a dependency of systemd, and the reason why systemd works only with linux, which in turn is one of the reasons why I think systemd is not a sensible default for the “Universal Operating System”.
I do NOT agree with Mr. Preining’s assessment of systemd- but I concede that he is more conversant with the software than am I, even though I use it every day. He’s an actual Debian Developer who had an actual vote in the GR. Who the hell are you, and what standing do you have to tell Mr Preining, who is part of and helps to build Debian, to “try pull requests” to a software project he neither likes nor endorses?
Service management was not onerous before, even for those of us who manage fleets of servers. If you’re using config management you’ll find that (for example) Ansible abstracts it quite nicely [1]. As to the other “convenience features” mentioned, what are they specifically? I don’t know how to configure the clock nor locales using systemd- in Debian both those tasks are handled by dpkg-reconfigure afaik.
Actually, now I’ve had a look I can see that `localectl` manipulates locale settings for systemd.
None of ‘clock’, ‘ctl’, nor ‘systemd’ in connection with `man -k` showed me any way of configuring the clock via systemd. I’d be interested if someone could point out the mechanism for doing so, or other fun systemd tricks.
[1] https://docs.ansible.com/ansible/service_module.html
full ack!
/j
> Try sending such bug reports to Linus and you’ll get a much worse response.
Nope – that is simply incorrect.
By the way, you do not “send bug reports to Linus” – apparently you are not educated and have not googled, so here is for you how to find out how to report bugs about linux:
https://bugzilla.kernel.org/
There you go buddy!
Hopefully you have learned something.
And by the way. I have had some interaction with both Pottering and Torvalds, and I would say that Torvalds is WAY more sympathetic, with WAY more empathy and humbleness than Pottering, which isn’t that hard.
I hade some questions about how the “modularity” of systemd, and got a very short and in-dignifying “answer”.
I made a rant in a mailing list about Linux would remove support for old SGI PC machines, as I hade one running some old things. Linus Torvalds mailed me and asked me if you needed the support still on the SGI Pc.
What is this example of?
It shows that Linus cares about the users of the software, more than they programmers. If you are behaving like Pottering, you will be out of the loop supporting Linux. As been shown by one of systemd main developers have been.
Mr Pottering and the systemd group only cares about the programmers, and the users are only an annoyance to the programmers. And if it is hard to fix, just mark it “will-not-fix”.
I mostly agree, too. But on the other hand: It is open source. If you do not like it, there is still the option to do it like the neo-vim guys. Make it “better” 🙂
> Make it “better”
Why should other people be obliged to fix up the crap that has been written there by others?
The real issue is not how awful systemd is though.
The real issue is why distribution developers forced users to accept it. There are no easy workarounds, excluding some that STILL do not work easily.
“It’s about a bug that simply does the equivalent of rm -rf / in some cases.”
Where “In some cases” is “in the cases where you tell it to do that”. The command ‘rm -rf /foo/.*’ is expanded (by the shell) to ‘rm -rf /foo/. /foo/.. […]’. In that case, rm (and systemd) has been asked to recursively remove (among other things) ‘/foo/..’, which is… ‘/’.
Typically, historically, if you told a Unix-ish system to do something stupid and dangerous, it would do something stupid and dangerous. A sharp tool is useful because it is sharp, but it is dangerous for the same reason. If you make it less dangerous, you make it less useful. Maybe you make it less useful in ways that you weren’t planning on using it anyway, but the ways you use a tool are not the only ways that everyone else uses that tool.
Or, to quote someone else explaining the sentiment as I’ve seen described many times: “Unix was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.”[0]
I am actually surprised that ‘rm’ will no longer delete paths ending in ‘..’. This was not always the case, and while I have never used the feature intentionally, I think special cases like that should be avoided if at all possible. The more regular and consistent a system is, the more predictable and powerful it is. And if, at some point in the future for some reason or another, I did feel the need to express “rm -rf foo” as “rm -rf foo/bar/..”, I would be most annoyed that rm prevented me from doing so. It’s *my* computer, and it should do as I tell it. Certainly, maybe put the behaviour behind an option similar to –no-preserve-root, but outlawing it entirely? Not convinced.
Donning my flame-proof garments, I will say that I would not be surprised if, sometime in the future, someone else who has already decided that systemd is the work of the Devil (and that Lennart is his prophet) actually wants “R! /foo/.*” to do what they told it to do, and starts ranting about how bad systemd is for mollycoddling them, and that how this proves Lennart thinks he is superior and knows better than the owner of the computer for preventing them from doing the thing they want to do.
🙂
[0] https://en.wikiquote.org/wiki/Unix
“I am actually surprised that ‘rm’ will no longer delete paths ending in ‘..’. ”
It does so since UNIX System III (1982). We now have 2017, BTW.
Ok, in V7 you had many more ways to shoot yourself in the foot, but even then you could not destroy the rootfs completely by issuing a ‘rm -rf /.*’ (hint: b/c of the KISS – keep it small & simple – philosophy rm(1) used to call rmdir(1), which was itself deleted at a very early stage), but not by an init(8) config setting.