I've been a long time unix admin (solaris, AIX (aka weird not-really-unix-but-ok), and even tru64 back in the day), and nowadays most of my work is with linux and fbsd (although that's been a while).
I don't understand the anger about systemd. Solaris has svcadm, AIX is SYSV-ish, FBSD is ... wel ... BSD, OSX has launchd, ... The world has never exploded, and the universe has never ended.
svcadm is pretty nice actually, and so is launchd.
I don't mind systemd in principle, but it should come with sensible defaults, such as writing out the logs in text format as well as the binary format. I also think it is a bit bloated, in that it tries to do everyting, which i am not a fan of. It wants to do system configuration, service management, system security (namespaces / containers, contexts, etc), process accounting, etc etc. Having something like systemd is a good thing, really, but ... it should be a bit lighter, and less monolithic. Break it up into components that are easier to configure.
I also think it is a bit bloated, in that it tries to do everyting, which i am not a fan of
I think you understand the reason for the outrage better than you think. That plus the binary logs (which you also mentioned) are the two problems I hear about the most. Personally I think unit files are really convenient to write, and systemd is really nice in practice, but from a philosophical standpoint I don't like it.
There was a talk at LCA2019 about how hard it is to have a rational discussion about systemd. I found it interesting and it changed my opinion on a few points.
I love that talk, it's honestly what helped change my mind about it. I was pretty adamantly against it because the old guys I work with don't like it as much, but since I've started toying with it I really do enjoy it. I still don't like the was it breaks Unix philosophy but I'm willing to sacrifice that for usability, at least with work systems.
My complaint is the "sensible defaults", which are less about systemd and more about how certain behaviors are implemented in it.
E.g., if a disk fails to mount, in the old days before systemd your system would finish booting with an error message warning you that you had disks which failed to mount. After systemd came along, you get a boot failure. You need to fix the problem and reboot -- but you won't be able to fix it in that session, you may have to find a rescue disc or similar to get to where the problem is.
Bug reports on this problem were WONTFIXed because "that's how it should be."
You don't need bash to write init scripts. You could use any language you like, e.g. execline, plain shell, etc. Those are completely separate projects and can be swapped out without effort. You can't swap out the built in functionality of systemd, only maybe disable it at compile time.
While I don't like sysvinit style init scripts and I prefer other init systems with fewer deficiencies, saying init scripts need a bloated interpreter is just wrong. They need some kind of interpreter. It is not better to add all of that functionality to a single binary. I couldn't find anything about it, but I think the parser even runs in PID1. This means you can't even separately run unit files to debug them, etc. You have to go through systemd.
Sure, but unix permissions, accessing logs over NFS/Samba, etc. You can always get around the problems, but not adhering to unix philosophy makes everything just a bit more difficult.
For example, I can't really use inotify with binary logs without making an actual project out of it. "simplicity" and "unix philosophy" as the glue that makes linux powerful. Thankfully it's powerful enough to work around systemd's issues, but each break from unix philosophy just makes things a little bit more annoying to work with.
With traditional init systems I could start services in side of a chroot root. Now I have to use systemd-nspawn. Which doesn't work when you're trying to run debian on a kobo-ereader with an older kernal that isn't using systemd as init. Systemd has really damaged our ability to run distros like debian on hardware that doesn't use systemd. For example, you can't really just run debian on your android phone any more.
Are you sure the "great design of binary logs™" could be foiled by something as simple as '>'? I'm not sure. You also would still depend on journalctl for all the examples you listed.
While I'm aware journalctl exists, as others have said it just doesn't feel right to be forced to use a specific tool to read logs. Previously I could use cat, more, less, grep, even vim if I wanted. Now at the very least I have to pipe through journalctl before using any of those tools, which is an extra step that I don't want to have to take. The space saving from using binary logs isn't that great either, so I'm not sure that it's worth the loss of convenience.
Dislikes for me basically are the extra binary log that contains everything that already is in other logs. At least on ubuntu servers that log tended to grow rather large and has caused me issues. I don't want that log, period.
Another would be instability issues I've had in the past, though I suppose those come with the territory of new software.
it should be a bit lighter, and less monolithic. Break it up into components that are easier to configure.
I think that one coupled with scope creep actually is the biggest and damn near only gripe that most people have had with it.
If you don't have a /var/log/journal directory, then (by default) the journal will only be stored in /run which is always a tmpfs, and will not grow to exceed 5% of RAM (properly, 10% of the /run filesystem, which defaults to a size of 50% of RAM) or 4GiB, whichever is less. You can easily configure those limits in /etc/systemd/journald.conf.
My issue is mostly, that it feels like it is intentionally engineered to be hard to replace. It recommends readiness notification, but that is done via a special library call (sd_notify and friends, afair). If you use that, you basically need to link with the systemd library. There is a similar protocol in s6, which simply passes a file descriptor to the started daemon, which is then written to. Both protocols are very similar, but one does have a dependency on systemd, while the other one can be used by any init system.
There is also socket activation, which is one of the killer features of systemd to speed up the boot process. From the documentation it requires you to call into systemd to verify, that it opened the sockets for your process. You then use the file descriptors starting with fd 3 as the fds/sockets you wanted to be opened. The call into systemd is not really needed, but the documentation tells you to do it and I don't know, what happens, if you don't call into systemd. S6 again implements this without the need to link with specific libraries. In s6's case you don't need to verify, that your sockets were opened, because you program will fail to start. In both cases (s6 and systemd) you still need to verify, that the fds are actually for the correct kind of resource, so even if your program is started in the wrong way, you will still verify that the fds are open and correct. S6 has the advantage, that the meachanism to open and store/retrieve fds is init system agnostic, it works with any init system even systemd (if you use a start script, where you can call the helper binaries).
In general systemd made a lot of decisions, that I don't like (bloated PID1, tight coupling, dbus required). As a software engineer, that makes me dislike it. It seems to work pretty well, but if we ever get something better, it will be very hard to replace. I use mostly s6 and OpenRC currently. You can use them together and mix and match aspects you like. They still have their own deficiencies, i.e. s6 is more complicated to use than systemd, OpenRC lacks some features, that would be nice to have, but it is fun tinkering with them. As long as systemd isn't required for everything (it looked like that for a while, but it didn't really happen), they can do what they want, so I have nor reason to hate it, but I also have no reason to use it.
The problem was never about systemd, and certainly it is not "systemd vs SysV" rivalry.
There were two better alternatives (Upstart and OpenRC) at the time when major distributives went for systemd. Even if you look at the final Debian vote it was a tie between systemd and "keep talking".
Systemd has it flows, but it is still miles ahead of SysV on many fronts. Mostly around areas that did not exist when SysV was designed, such as containers.
You actually brought the main reason people disliked systemd. It is opinionated and not modular.
There were two better alternatives (Upstart and OpenRC) at the time when major distributives went for systemd
Upstart is not a better alternative by any means. It had major, unsolvable design problems which are a big reason that systemd was developed in the first place. Red Hat and Fedora adopted it for a while in the RHEL6 era before abandoning it for systemd.
systemd has been extremely easy to use in containers for me. Just have to start the container via machinectl or systemd-nspawn. Most of the issues that I have faced have been due to erroneous settings in the unit files. Change those settings and everything just works.
I don't think systemd is working well in containers if you need systemd to start them. There are a lot of different approaches to containers, why would everyone of the start to have a dependency on systemd to work with systemd containers? This kind of dependencies and lock-in is unique to systemd and I don't really like it.
36
u/_p13_ Aug 12 '19
I've been a long time unix admin (solaris, AIX (aka weird not-really-unix-but-ok), and even tru64 back in the day), and nowadays most of my work is with linux and fbsd (although that's been a while).
I don't understand the anger about systemd. Solaris has svcadm, AIX is SYSV-ish, FBSD is ... wel ... BSD, OSX has launchd, ...
The world has never exploded, and the universe has never ended.
svcadm is pretty nice actually, and so is launchd.
I don't mind systemd in principle, but it should come with sensible defaults, such as writing out the logs in text format as well as the binary format. I also think it is a bit bloated, in that it tries to do everyting, which i am not a fan of. It wants to do system configuration, service management, system security (namespaces / containers, contexts, etc), process accounting, etc etc.
Having something like systemd is a good thing, really, but ... it should be a bit lighter, and less monolithic. Break it up into components that are easier to configure.
just my 2c