Timeline for Why are POSIX mandatory utilities not built into the shell?
Current License: CC BY-SA 3.0
28 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 19, 2024 at 23:56 | vote | accept | Kyle | ||
| Mar 3, 2017 at 21:13 | comment | added | jlliagre | @Joshua Got it, stripped down at link time. Thanks! Being smaller also makes them withstand disk corruption even better. | |
| Feb 27, 2017 at 16:15 | comment | added | Joshua | @jlliagre: ld garbage collects when linking. | |
| Feb 27, 2017 at 8:50 | comment | added | jlliagre | @Joshua Your first statement doesn't make sense so I either you are confusing terms or the /sbin binaries are statically linked with a stripped down libc. Quoting HP-UX admin guide /sbin: Contains statically linked versions of critical programs needed at boot time or when important shared libraries have become corrupted. Rescue boot doesn't generally requires an external media on HP-UX and so is done by using single-user mode, but when that one fails, CD/DVD, tape and network boot are obviously possible with HP-UX like any other Unix, might require an extra piece of software (Ignite-UX?) | |
| Feb 27, 2017 at 1:34 | comment | added | Joshua | @jlliagre: It takes several hundred statically linked binaries to weigh the same as the standard library. The fact that they're duplicates of part of the shared library is irrelevant because the shared library is in /usr not /. Rescue boot = boot from removable media to repair the existing system rather than copy a new system over it. | |
| Feb 26, 2017 at 21:51 | comment | added | jlliagre | @Joshua You misunderstood why I talk about disk issue resiliency. Standalone binaries can directly run without any prerequisite, like extra file to load or dynamic loader configuration settings, so they are inherently more resilient to disk issues. Statically linked binaries do not help keeping a file system small. On the opposite statically linked binaries are bigger because they need to embed the libraries they use. Not sure about what you mean with "no rescue boot". You can rescue the system from single-user mode, don't you? | |
| Feb 26, 2017 at 21:03 | comment | added | Joshua | @jlliagre: Indeed. /sbin/sh is not statically linked for disk issues. /sbin/sh is statically linked because /lib -> /usr/lib and the need to keep / small. (The stuff in /sbin uses only a small subset of libc ...). Also, HP-UX had no concept of rescue boot. | |
| Feb 26, 2017 at 17:43 | comment | added | jlliagre | @Joshua I guess they are needed for single-user mode and/or maybe /sbin/sh is statically linked so is more resilient to disk issues. I have no HP-UX system to check. | |
| Feb 26, 2017 at 15:15 | comment | added | Joshua | @jlliagre: HP-UX has /bin -> /usr/bin so all POSIX utilities needed for early boot are in /sbin. I'm pretty sure that everying POSIX says must live in /bin is duplicated there. Particular examples: echo, false, true, test, ulimit, :, [. There where quite a few more I can't remember off hand that were rather extensive shell scripts too. | |
| Feb 26, 2017 at 12:52 | comment | added | jlliagre | @Joshua, do you have examples? I'm not expecting POSIX utilities to be in /sbin. | |
| Feb 26, 2017 at 9:53 | history | edited | jlliagre | CC BY-SA 3.0 | added 358 characters in body |
| Feb 26, 2017 at 1:50 | comment | added | Joshua | In poking around HP-UX's /sbin; a lot of "standalone" utilities were shell scripts that just invoked the builtin. | |
| Feb 25, 2017 at 9:48 | history | edited | jlliagre | CC BY-SA 3.0 | added 8 characters in body |
| Feb 24, 2017 at 21:50 | comment | added | jlliagre | @slebetman Creating a subshell doesn't really spawn a new process. It just clones the existing one without using exec. This is much lighter due to the copy on write fork used by Linux and Unix implementations, as jpaugh already commented. | |
| Feb 24, 2017 at 19:56 | comment | added | jpaugh | @slebetman In modern OSes, fork without exec can take advantage of Copy-on-Write, which can be much faster. | |
| Feb 24, 2017 at 19:45 | comment | added | slebetman | @jlliagre: OK. Then I don't understand why you'd say subshells are more lightweight than processes when a subshell spawns a process. | |
| Feb 24, 2017 at 13:04 | comment | added | jlliagre | @slebetman Why are you again introducing threads in this discussion while they are irrelevant here? As far as I know, no mainstream shell is exposing or using multi-threading. Not only on Linux but on whatever OS implementing it, a successful fork creates a process. This process is running the very same command as its parent which is often useless outside the specific case of subshells. That's the reason why most child processes quickly call exec after their birth. | |
| Feb 24, 2017 at 12:26 | comment | added | slebetman | @jlliagre: On linux a fork creates a process. The point you're perhaps missing is that on Linux they've optimised processes so much that the developers have determined that there is not further advantage creating anything more lightweight. Basically in linux a process is as lightweight as a thread. | |
| Feb 24, 2017 at 10:08 | comment | added | sourcejedi | @Chunko note my test compared echo >/dev/null with cat </dev/null. Busybox was able to nofork the echo command, despite the redirection. (It backed up stdout to another FD, then restored it afterwards). | |
| Feb 24, 2017 at 9:57 | comment | added | sourcejedi | I measured busybox nofork builtins as having on the order of 10x less overhead than noexec builtins, which in turn had ~5x less overhead than fork+exec of a separate binary. Definitions as per unix.stackexchange.com/a/274322/29483 It's interesting that busybox doesn't nofork everything, although I know some busybox code is shortened by not cleaning up memory, and just relies on being a short-lived process. | |
| S Feb 24, 2017 at 8:31 | history | suggested | Bakuriu | CC BY-SA 3.0 | Added quote block for quotation. |
| Feb 24, 2017 at 8:30 | comment | added | jlliagre | @slebetman You are missing my point. Subshells are neither threads nor exec'ed processes, regardless of whether they are running on Linux or not. Subshells are just their parent's clone, created by a fork not followed by exec; fork is nowadays a very lightweight operation compared to exec. | |
| Feb 24, 2017 at 7:53 | comment | added | slebetman | @jlliagre: Depends on the OS. On linux there's very, very little difference between threads and processes. Indeed, unlike other unixen linux don't have separate implementations of process and threads. On linux you can start with a process and close enough resources (stdio etc) to become a thread and you can also start with a thread and open enough resources to become a process. They're the same thing only with different default start state. | |
| Feb 24, 2017 at 7:42 | review | Suggested edits | |||
| S Feb 24, 2017 at 8:31 | |||||
| Feb 24, 2017 at 2:47 | comment | added | jlliagre | @Chunko Yes. subshells are lighter than fork/exec'ed processes though. | |
| Feb 24, 2017 at 1:32 | comment | added | Chunko | This is the right answer, but I would just add that as the interface of these utilities is generally via stdin / stdout anyway, that even if every one of them were also implemented as a built-in routine in bash, it would effectively still need to fork itself and create pipes for each command in a pipeline anyway, so there would be only marginal gains | |
| Feb 24, 2017 at 1:11 | history | edited | jlliagre | CC BY-SA 3.0 | added 583 characters in body |
| Feb 23, 2017 at 22:48 | history | answered | jlliagre | CC BY-SA 3.0 |