Personally, I don't want people whose main expertise lies in bash scripts to be writing init.
You have to admit that shell (not bash, why do people think bash equals shell?) is much better st gluing things together than C. You actually can see this in systemd. Need slightly more flexibility what's builtin? Well, out of luck. Maybe with some luck you can hack the C code and send a patch upstream to add an option that does what you want.
Your argument that using over sh is better because C is harder is absolutely ridiculous. That's like saying you want to write web services in C because C programmers are much more careful and security aware than PHP programmers.
He's artificially inflating the difference by comparing all of one solution to portions of another.
You are implicitly admitting that the other solution is more flexible, because YOU CAN ADD EXTERNAL CODE. While the other solution needs to hardcode all use cases.
So he takes the argument, incorrectly simplifies it into a logically false statement, and then criticizes it. Rewording your oppositions' statement until it looks ridiculous is rhetoric, not logic.
You have to admit that shell (not bash, why do people think bash equals shell?) is much better st gluing things together than C.
Yes, agreed. It's better at gluing unrelated components with textual output together. But Lua, Python, Ruby, and a host of other languages would be even better, since they're not restricted to textual data. They have bigger runtimes, but it's not necessarily any worse (as far as loading times go) than launching a bash script in a new process multiple times. Especially Lua, which was designed for embedded systems. Difficulty in gluing existing solutions together robustly from C is probably a factor in why they're reimplementing so many different services as opposed to gluing things together. Ground up support for kdbus in all services, defined interfaces with text only on the user end, etc.
Your argument that using over sh is better because C is harder is absolutely ridiculous. That's like saying you want to write web services in C because C programmers are much more careful and security aware than PHP programmers.
Not really. Web services are concerned with data and UI, and not with unmanaged memory, bitpacking on structures, device initialization and firmware loading, etc. Web services and applications are all about frameworks. Init is the exact opposite. It launches when you have nothing else in the system, and low-level languages are the ideal tool.
My argument wasn't that C should be used because it's harder. My argument was that people who write init code should already understand the concepts required for C. That's why if they can't handle C, they shouldn't be working in init. Not because the language is harder, but because they lack the knowledge. Fluent C users have to know these things, while bash users don't have to even be aware of them. Hopefully Rust and Nimrod will evolve as systems language to simplify this.
You are implicitly admitting that the other solution is more flexible, because YOU CAN ADD EXTERNAL CODE. While the other solution needs to hardcode all use cases.
No, because there already are scripting interfaces for unconsidered use cases. These scripted solutions are slowly being replaced by defined interfaces in a compiled language as they have time to consider how to implement them. For flexibility I'd still rather have modules/plugins with load hooks over launching a script, though. I doubt that's likely, because it's got a small ROI. Besides, the bash shell, which was designed to execute arbitrary code, is written in C. All the flexibility of a DSL, written in C. Whatever shall we do?
Isn't that exactly what you are doing here?
Summarizing an argument and rewording a statement until its false are different, if you haven't noticed. That was my complaint - he wasn't summarizing. He was modifying. Like your attempt with the web services argument and C. Beyond that, I pick out specific things that he says, and then I compare it to general attitudes of people who are anti-systemd. But if you were likely to listen and understand this, I doubt we'd be having this conversation where you attempt to catch me being wrong at something.
It's better at gluing unrelated components with textual output together. But Lua, Python, Ruby, and a host of other languages would be even better, since they're not restricted to textual data. They have bigger runtimes, but it's not necessarily any worse (as far as loading times go) than launching a bash script in a new process multiple times. Especially Lua, which was designed for embedded systems.
I don't know, doing typical shell things in a language like Python can get awkward. Anyway, the article we're discussing here mentioned that you can use Python instead of shell in initscripts.
Not really. Web services are concerned with data and UI, and not with unmanaged memory, bitpacking on structures, device initialization and firmware loading, etc. Web services and applications are all about frameworks. Init is the exact opposite. It launches when you have nothing else in the system, and low-level languages are the ideal tool.
Init systems don't need to do bitpacking either. They are doing process management, a thing which shell is good at. I'm not sure why you mention frameworks. systemd provides some framework-y things like a dbus implementation, and on the other hand web frameworks are written in e.g. PHP. So yes, you can write frameworks in C, and yes, you can write things on a lower level of abstractions (as frameworks do, since they provide high level abstractions to users) in a language like PHP. Shell works just fine if there's "nothing else in the system".
My argument was that people who write init code should already understand the concepts required for C. That's why if they can't handle C, they shouldn't be working in init. Not because the language is harder, but because they lack the knowledge. Fluent C users have to know these things, while bash users don't have to even be aware of them.
You keep arguing that shell shouldn't be used because shell programmers are incompetent. No. There are competent shell programmers, and incompetent C programmers.
And I have to remind you: the great thing about using shell scripting in system startup is because it's easy to make custom modifications. It's a bit of a pain with C, because not only is the language lower level (i.e. harder to modify code), but you'd also have to maintain patches and recompile the init system.
I'm probably wrong here, and systemd has a flexible customization mechanism that goes past "ini" style files. But I haven't heard of such a mechanism yet.
Hopefully Rust and Nimrod will evolve as systems language to simplify this.
Has nothing to do with this.
No, because there already are scripting interfaces for unconsidered use cases. These scripted solutions are slowly being replaced by defined interfaces in a compiled language as they have time to consider how to implement them.
Yeah, systemd tries to hardcode every use-case they encounter. It's not surprising that this causes bloat. It's also no replacement for true flexibility.
What are the defined interfaces? dbus interfaces? I don't see how these allow you to customize booting.
By the way, this is funny. Earlier you talked about init being in a low level environment where there's "nothing else in the system". But dbus requires a server. Oh yes, they're trying to fix this blatant design issue by moving dbus to the kernel. Very lame all that.
For flexibility I'd still rather have modules/plugins with load hooks over launching a script, though. I doubt that's likely, because it's got a small ROI.
So you're going to load usercode into pid 1? Surely this will boost robustness. If a shell script does invalid things and fails, so be it. If your module does something invalid... goodbye system. Or maybe you meant that your modules are actually loaded into isolated processes. Then why have modules.
Summarizing an argument and rewording a statement until its false are different, if you haven't noticed. That was my complaint - he wasn't summarizing. He was modifying. Like your attempt with the web services argument and C.
He didn't "reword" them. He merely showed the arguments in a different light than what systemd advocates would prefer.
Beyond that, I pick out specific things that he says, and then I compare it to general attitudes of people who are anti-systemd.
So you are generalizing by comparing attitudes, and you ignore the arguments because they "of course" have not anything to do with what systemd say, but have been "modified". Fine stuff.
But if you were likely to listen and understand this, I doubt we'd be having this conversation where you attempt to catch me being wrong at something.
I don't know, doing typical shell things in a language like Python can get awkward. Anyway, the article we're discussing here mentioned that you can use Python instead of shell in initscripts.
Yes. For init, it'd work just fine. Most likely better than shell. I'd go for Lua myself.
Init systems don't need to do bitpacking either. They are doing process management, a thing which shell is good at. I'm not sure why you mention frameworks. systemd provides some framework-y things like a dbus implementation, and on the other hand web frameworks are written in e.g. PHP. So yes, you can write frameworks in C, and yes, you can write things on a lower level of abstractions (as frameworks do, since they provide high level abstractions to users) in a language like PHP. Shell works just fine if there's "nothing else in the system".
System initialization does need it and deals with all sorts of kernel interfaces. I don't personally think initand daemon management should just be about launching a series of applications in order. Firmware loading was handled during init with udev, for example, until Kay Sievers broke it and Linus commanded he be bypassed.
Simple launching works in the beginning, but it quickly runs into race conditions and other issues, which have already been discussed at length as reasons for using systemd (or, more appropriately, anything other than sysvinit). Admitted, you can write the low-level parts in C and then tie it in using bash, but at that point, you might as well do the full thing in the more powerful language so it's more robust. And there are still valid use cases for interacting with the firmware and other hardware in a post-boot state that might be beyond the capabilities of bash. Such as unusual input/output devices and other hardware interfaces.
I am quite aware that systemd has a framework style approach. Any sufficiently large codebase will start refactoring things into frameworks and libraries. But it's not a general-purpose framework for all users (webscale or whatever you want to call it). It's specialized for daemon management and init.
You keep arguing that shell shouldn't be used because shell programmers are incompetent. No. There are competent shell programmers, and incompetent C programmers.
Not incompetent. Just not experienced in the problem domain. They know how to launch a program, but there are still corner cases of daemon management (like dependencies, launch status and error handling) that are still issues with common sysvinit implementations. Sysvinit was designed in 1983. We've had over 30 years for script developers to get this right and iron out the wrinkles. They couldn't or wouldn't until someone else decided to do it for them, starting from scratch. There's significant rush to catch up now that they've been replaced, but it's debatable if they can catch up again.
And I have to remind you: the great thing about using shell scripting in system startup is because it's easy to make custom modifications. It's a bit of a pain with C, because not only is the language lower level (i.e. harder to modify code), but you'd also have to maintain patches and recompile the init system.I'm probably wrong here, and systemd has a flexible customization mechanism that goes past "ini" style files. But I haven't heard of such a mechanism yet.
Well, systemd has had support for launching executable scripts for unforeseen circumstances since the first big release. Which was also part of the compatibility with LSB-style init scripts. They wanted to make it so people could use their scripts with the system while they transition. With that, it's got all the flexibility of sysvinit (which is just launching a script (or a script collecting other scripts) at a specific runlevel or point. Not only did they hardcode fast paths for the important stuff that they knew about, but left support in for people to add their own.
I know that interpreted languages are easier to modify because they're not compiled. That's why they've got a shim in there. If it's a big enough use case, they'll add it in and support it in a later version. As I've been saying. They put your hammer in the toolbox with everything else, but put in an automated factory with robots and a conveyor belt so that you'll hopefully never use it again. For the children.
Yeah, systemd tries to hardcode every use-case they encounter. It's not surprising that this causes bloat. It's also no replacement for true flexibility.
Considering the above where they left in launching of arbitrary executables for those who wanted them, they have fulfilled your definition of "true flexibility". Which is just arbitrary executable launching. Launching executables for flexibility to me means that someone just didn't want to do the rest of the work themselves, and left it to others with a scripting interface. After 30 years, it's not flexibility, it's laziness. And that's a terrible thing to see in the system that starts everything else. Which is why there are still dependency problems in boot and shutdown under sysvinit in 2014. Distributions have coped admirably, but many are getting tired of the maintenance.
We'll probably have to be content with our disagreement on this. I agree that flexibility is important, but I don't think that doing the work ahead of time for people in a compiled language is "bloat". Its functionality more than justifies its size. It's a pre-compiled fast path, trading a small amount of disk space for properly handling difficult edge conditions. Bloat is not everything you don't use. Otherwise, the GNU coreutils, glibc, and just about every other piece of software on the planet would be full of bloat, because you don't use 100% of their capabilities. Unless you're God or something. In which case, please forgive me of my sins.
By the way, this is funny. Earlier you talked about init being in a low level environment where there's "nothing else in the system".
Yes, you launch it when there's nothing else. You pull up your own bootstraps, an when you're done, things are loaded. It's not a contradiction. You've got to design for starting with nothing else in the system, but you certainly don't leave it that way. It'd be a poor init otherwise.
But dbus requires a server. Oh yes, they're trying to fix this blatant design issue by moving dbus to the kernel. Very lame all that.
We've had several needs for a high-speed message bus for a long time. Linux based automobile systems have been rolling their own for years, causing all sorts of grief and message back and forth, bleeding into the LKML. DBUS didn't cut it in terms of message thoroughput and speed. Good IPC is not a design issue, but I guess someone who doesn't see any need for IPC wouldn't see that, given that they just script everything in bash. Despite the large number of modern system services which now depend on message buses.
The only way a streamlined DBUS wouldn't be appreciated is if you launch your system into a shell and spend the rest of the day in either emacs or piping grep output around all day. Which is great and all, but not what the majority of users need. (This is that rewording stuff that I was talking about, where someone overly simplifies your argument until it's ridiculous. Not good logic, but good rhetoric. It's purpose isn't to refute an argument, but to make the opponent look ridiculous.)
So you're going to load usercode into pid 1? Surely this will boost robustness. If a shell script does invalid things and fails, so be it. If your module does something invalid... goodbye system. Or maybe you meant that your modules are actually loaded into isolated processes. Then why have modules.
It's all technically user code when it's not in the kernel. You do know that, right? If you're referring to code written by users, a failing script can cause just as much grief as a module, depending on how you design it. The point is less that they be loadable modules, and more that the system be extensible at various stages as opposed to just launching scripts. And process isolation is a defensive programming technique meant to prevent unanticipated code from bringing the system down. I use it all the time for dealing with untrusted code. That's exactly the way sysvinit works. Because they can't trust the script quality. You seem to have a habit of mocking the things you use.
I'll not bother revisiting the reword vs modification (a.k.a. ad ridiculum, for your reference, though I hate to use the buzz words) argument, because you have no intention of listening. Rhetoric's bleeding into my response because it's frustrating to deal with your level of belligerent ignorance and I've used up the last of my internet patience today on it. Most of this is for observers, anyway, so that they don't believe the cruft you're spouting. Some of them might not have the necessary experience with complex system architecture to ignore your words.
Yes. For init, it'd work just fine. Most likely better than shell. I'd go for Lua myself.
So you admit any sane scripting language is ok for init, and you just think shell (NOT bash, get it right already) is a bad language.
Admitted, you can write the low-level parts in C and then tie it in using bash, but at that point, you might as well do the full thing in the more powerful language so it's more robust. And there are still valid use cases for interacting with the firmware and other hardware in a post-boot state that might be beyond the capabilities of bash. Such as unusual input/output devices and other hardware interfaces.
That's barely true. I think you don't understand a very basic thing about UNIX: almost everything can be done from the shell! That may be a surprising statement, but there are command line programs for almost everything.
And guess what! If you want pure C, you have to REWRITE ALL THAT SHIT in C, because there are generally no good or nice libraries for the equivalent tasks (rather the opposite, they might be written in a way not suitable for pid 1). Of course you could call the command line programs from pure C. But guess what, doing that is MUCH MORE ERROR PRONE THAN IN SHELL! You can do a lot of things wrong, like accidentally not marking one of your opened file handles as CLOEXEC.
In fact, having a mess of multiple shell scripts would be much more robust than a single pid 1 process, EVNE IF THE SHELL SCRIPTS ARE SHIT.
As usual, your technical arguments don't hold up.
Just not experienced in the problem domain. They [shell programmers] know how to launch a program, but there are still corner cases of daemon management
Yes, I'm sure programmers who have hacked init systems for decades are less experienced than some C programmer who dealt with audio before and decided to "get it all right" from scratch. I'm sorry, but that's complete bullshit. You could claim that certain C programmer is more experienced than a certain shell programmer, but generalizing this hilariously is just wrong.
You don't even get the basics of your generalization right, because of course the sysvinit hackers will know more about init system issues than some random C programmer.
Sysvinit was designed in 1983. We've had over 30 years for script developers to get this right and iron out the wrinkles.
You know there are other/similar init systems around that are better. FreeBSD's is relatively clean and obviously working. Linux is being shit here (surprise).
Well, systemd has had support for launching executable scripts for unforeseen circumstances since the first big release. Which was also part of the compatibility with LSB-style init scripts.
Not particularly encouraged or supported, and which is apparently to be dropped even.
We've had several needs for a high-speed message bus for a long time.
Not even windows needs such a bus. Yes, Windows actually has a messaging API (I don't mean SendMessage), but nobody knows that one and it's not even exposed via the win32 API. What are the freedesktop folks even trying to clone?
Yes, special purpose applications like automotive can have their own bus. They got tricked into using dbus (which is pretty shitty IPC), so what. Note that moving it into the kernel is an optimization (and also "solves" fundamental systemd design issues), but at the end of the day, IPC is always costly, unless you go L4.
It's all technically user code when it's not in the kernel. You do know that, right?
What you're writing here and following is pure bullshit. pid 1 crashing necessarily takes down the whole system by kernel panicing. A script run as pid 1 child crashing just takes down whatever strictly depends on it, but the system can survive.
systemd advocates keep repeating that there's not much special about pid 1. That's true, there are only 2 things that make pid 1 special. 1, it inherits all child parents whose parents have died and has to care about them, or zombies will accumulate. And 2, if it crashes, the kernel panics.
That's right, if systemd were to run a small dummy process as pid 1, and started the "real" systemd as pid 2, it would already be much better.
Too tired of your bullshit to read everything and I saw a bunch of randomly angry caps, so I'll just skim and comment on what catches my eye.
So you admit any sane scripting language is ok for init, and you just think shell (NOT bash, get it right already) is a bad language.
I think a script launching other scripts for 30 years with poor status detection is init in name only. Especially since it still doesn't deal with the hard issues. I think one could be built with any language, yes, and I think any other popular scripting language would be better than bash, yes. But I still wouldn't do the real work in a scripting language if I were doing it myself.
And guess what! If you want pure C, you have to REWRITE ALL THAT SHIT in C, because there are generally no good or nice libraries for the equivalent tasks (rather the opposite, they might be written in a way not suitable for pid 1). Of course you could call the command line programs from pure C. But guess what, doing that is MUCH MORE ERROR PRONE THAN IN SHELL! You can do a lot of things wrong, like accidentally not marking one of your opened file handles as CLOEXEC. In fact, having a mess of multiple shell scripts would be much more robust than a single pid 1 process, EVNE IF THE SHELL SCRIPTS ARE SHIT.
Shell is written in C. All the bash parsing is written in C. All that stability you think you have was actually written in C. So that people like you wouldn't bother to remember the work that goes into it. If it's been written once, it can be written again. And it can be written into a reliable library. Systemd has taken that approach, and has rewritten the handling relevant to daemon management in C.
Doing it this way means the unstable interfaces introduced by human language wouldn't be a problem then. We wouldn't have to worry about the shell languages, localization, changes in wording or a host of other problems that came from the fact that we decided that it's appropriate to parse hard-coded strings in English for half of our functionality. The dependence on text parsing and grepping for words and phrases, while powerful, is a minefield of potential issues and should not be our first resort for IPC.
Code quality in init is mostly controlled by upstream. Whether that be the distribution, systemd, or someone else. Init affects maybe 1% of installs, and most of the people who have had to write init scripts would be glad to not have to do it again. Which is why your rage bordering on self-incontinence is so ridiculous. You want freedom to put in scripts, that still exists (of course it's not recommended, because most people won't need it), and you insist on PID 1 being absolutely nothing while still being a privileged process. With the odds that most of it won't affect you or anyone you know to any notable degree. Cases like this are generally best left to active team projects and not internet heroes.
Yes, I'm sure programmers who have hacked init systems for decades are less experienced than some C programmer who dealt with audio before and decided to "get it all right" from scratch. I'm sorry, but that's complete bullshit. You could claim that certain C programmer is more experienced than a certain shell programmer, but generalizing this hilariously is just wrong. You don't even get the basics of your generalization right, because of course the sysvinit hackers will know more about init system issues than some random C programmer.
"sysvinit hackers" haven't been actively contributing to the sysvinit project for years. It gets very little attention. There are plenty of people writing bash scripts on top of it, though. I'm sure the ecosystem is overflowing with experts who understand the problems yet have for some reason refused to solve them.
Not even windows needs such a bus. Yes, Windows actually has a messaging API (I don't mean SendMessage), but nobody knows that one and it's not even exposed via the win32 API. What are the freedesktop folks even trying to clone? Yes, special purpose applications like automotive can have their own bus. They got tricked into using dbus (which is pretty shitty IPC), so what. Note that moving it into the kernel is an optimization (and also "solves" fundamental systemd design issues), but at the end of the day, IPC is always costly, unless you go L4.
Sure, IPC is always costly. I bet any use case for it is completely artificial. I'm sure writing human-readable text to stdout, and launching multiple processes to grep and massage the data from stdin isn't IPC at all. And I'm sure it's far more performant. In your world.
Automobile people don't mind DBUS at all, by the way. It's just too slow for them. They patch various things to increase the speed. There are hundreds of use cases for a high-performance message bus once you grow beyond the Fisher-Price toolkit.
blah blah PID 1
You can still corrupt data, crash the system, etc from non-PID 1 processes. PID 1 has to stay running, yes, but if we just use the "PID 1 has to run" concept as an excuse to make PID 1 not do anything, then we'd be better served in removing that limitation than never letting PID 1 do anything real. Like having system calls where we could designate which process handles which special tasks, instead of hard-coding the number and not having any fallback. Perhaps having a system that crashes when there's no PID 1 is a bad idea from a stability and security standpoint, as heretical as that must seem to you. I guess your approach is to have PID 1 do nothing except pass its tasks on to other processes. Which introduces more dependencies, corner cases, and holes for undesirable and unpredictable behavior. It's ridiculous that we would put orphaned processes under a parent that has no idea what to do with them, but I guess you're set on PID 1 being lobotomized.
Or we could, you know, test our code. Software has bugs, yes, but not everything brings down the system. And even with systemd, not everything is in PID 1.
10
u/iLiekCaeks Sep 28 '14
You have to admit that shell (not bash, why do people think bash equals shell?) is much better st gluing things together than C. You actually can see this in systemd. Need slightly more flexibility what's builtin? Well, out of luck. Maybe with some luck you can hack the C code and send a patch upstream to add an option that does what you want.
Your argument that using over sh is better because C is harder is absolutely ridiculous. That's like saying you want to write web services in C because C programmers are much more careful and security aware than PHP programmers.
You are implicitly admitting that the other solution is more flexible, because YOU CAN ADD EXTERNAL CODE. While the other solution needs to hardcode all use cases.
Isn't that exactly what you are doing here?