4

After having opened the master part of a pseude-terminal

int fd_pseudo_term_master = open("/dev/ptmx",O_RDWR);

there is the file /dev/pts/[NUMBER] created, representing the slave part of he pseudo-terminal.

Ignorant persons, like me might imagine that after having done ptsname(fd_pseudo_term_master,filename_pseudo_term_slave,buflen); one should be set to simply do int fd_pseudo_term_slave = open(filename_pseudo_term_slave,O_RDWR); and be good.

However there must be a very important use case of "locked" pseudo-terminal slaves, since to make stuff simple, before the open call can be made, it is made necessary to use man 3 unlockpt, to "unlock it".

I was not able to find out what this use-case is? What is the need of the pseudo-terminal to be initially locked? What is achieved with code (taken from a libc)

/* Unlock the slave pseudo terminal associated with the master pseudo
   terminal specified by FD.  */
int
unlockpt (int fd)
{
#ifdef TIOCSPTLCK
  int save_errno = errno;
  int unlock = 0;

  if (ioctl (fd, TIOCSPTLCK, &unlock))
    {
      if (errno == EINVAL)
        {
          errno = save_errno;
          return 0;
        }
      else
        return -1;
    }
#endif
  /* If we have no TIOCSPTLCK ioctl, all slave pseudo terminals are
     unlocked by default.  */
  return 0;
}

If possible an answer would detail a use-case, historical or current.

Bonus part of the question would be:

Do current linux kernels still rely on this functionality of "locked pseudo terminal slaves?"

Idea: Is this an inefficient attempt to avoid racing contitions?

Waiting for an answer I have looked more into the linux kernel source without having any good answer myself. However it appears that one use that can be "extraced" from an initial lockdown case of the pseudo-terminal is to provide some time for the pseudo-terminal-master process to setup some access rights to the file at /dev/pts/[NUMBER], as to prevent some user to access the file in the first place. Can this be part of the answer? Strangly then, however it appears that such "initial lockdown" state seems not really able to prevent multiple openings of the slave file anyway, at least to what I conceive to be guaranteed atomicity here.

mtraceur
  • 1,146
  • 9
  • 14
humanityANDpeace
  • 13,722
  • 13
  • 61
  • 107
  • fwiw, `unlockpt()` is a NOP on *BSD (it's just checking if the passed fd is a tty), and on linux it's just clearing a flag that prevents opening, nothing more. (`TTY_PTY_LOCK` via `ioctl(TIOCSPTLCK)`). So any portable program cannot rely on `unlockpt()` to do anything at all. It's probably just a leftover from the old-style (non-multiplexed) pseudo-tty implementation (just like `grantpt()`). –  Oct 23 '18 at 12:36
  • @mosvy I read the manpage on `grantpt` which tells its a "leftover" and indeed also `unlockpt` seemed somewhat of an cargo-cult anoyance. So it might be obsolete, sure and I have no insight or knowledge about the "old-style"(non-multiplexed) ptty you mention. However given it some more though, a process offering a ptty might be interested that only certain other processes can access this resource (e.g. because input from the ptty slave is evaluated with difference priveledges). This thinking made me change the title to reflect this aspect. – humanityANDpeace Oct 23 '18 at 12:55
  • 1
    I already answered this at https://unix.stackexchange.com/a/470853/5132 . – JdeBP Oct 23 '18 at 13:33
  • 1
    @JdeBP read your answer and indeed it provides much of the explanation I would have sought here, that is great. Basically it confirms that the `unlockpt` is only left over "cargo-cult" stuff that is not anymore necessary on BSDs. Have to reread it again to get if that is true for Linux kernels too. I would like to link to your answer, but I do not really feel that the specific question here is a duplicate of the downvoted one your answer is attached to. if you feel like giving a succinct answer here I would appreciate that :) – humanityANDpeace Oct 23 '18 at 14:36
  • 2
    @humanityANDpeace In the old style pty implementation, there was no `/dev/ptmx`, but some 50 `/dev/ptyp0`, `/dev/ttyp0`, `/dev/ptyp1`, etc. pairs of device nodes (I hope I remember the names right), and you had to try the masters (`/dev/ptypXX`) until you found one that was free, open it, get the name of the slave from it by changing `p` to `t`, set the right perms on the slave (which was usually done by forking and exec'ing a setuid program and wait for it to terminate), and "unlock" the tty after all this had succeeded. The `grantpt()` and `unlockpt()` were convenient wrappers around that. –  Oct 23 '18 at 17:38

1 Answers1

6

The old AT&T System 5 mechanism for pseudo-terminal slave devices was that they were ordinary persistent character device nodes under /dev. There was a multiplexor master device at /dev/ptmx. The old 4.3BSD mechanism for pseudo-terminal devices had parallel pairs of ordinary persistent master and slave device nodes under /dev.

In both cases, this meant that the slave device files retained their last ownership and permissions after last file descriptor closure. Hence the evolution of the grantpt() function to fix up the ownership and permissions of the slave device file after a (re-used) pseudo-terminal had been (re-)allocated.

This in turn meant that there was a window when a program was setting up a re-used pseudo-terminal between the open() and the grantpt() where whoever had owned the slave device beforehand could sneak in and open it as well, potentially gaining access to someone else's terminal. Hence the idea of pseudo-terminal slave character devices starting in a locked state where they could not be opened and being unlocked by unlockpt() after the grantpt() had been successfully performed.

Over the years, it turned out that this was unnecessary.

Nowadays, the slave device files are not persistent, because the kernel makes and destroys things in /dev itself. The act of opening the master device either resets the slave device permissions and ownership, or outright creates the slave device file afresh (in the latter case with the slave device file disappearing again when all open file descriptors are closed), in either case atomically in the same system call.

  • On OpenBSD, this is part of the PTMGET I/O control's functionality on the /dev/ptm device. /dev is still a disc volume, and the kernel internally issues the relevant calls to create new device nodes there and reset their ownerships and permissions.
  • On FreeBSD, this is done by the posix_openpt() system call. /dev is not a disc volume at all. It is a devfs filesystem. It contains no "multiplexor" device nor master device files, because posix_openpt() is an outright system call, not an wrapped ioctl() on an open file descriptor. Slave devices appear in the devfs filesystem under its pts/ directory.

The kernel thus ensures that they have the right permissions and ownership ab initio, and there is no window of opportunity where they have stale ones. Thus the grantpt() and unlockpt() library functions are essentially no-ops, whose sole remaining functionality is to check their passed file descriptor and set EINVAL if it isn't the master side of a pseudo-terminal, because programs might be doing daft things like passing non-pseudo-terminal file descriptors to these functions and expecting them to return errors.

For a while on Linux, pseudo-terminal slave devices were persistent device nodes. The GNU C library's grantpt() wasn't a system call. Rather, it forked and executed a set-UID helper program named pt_chown, much to the dismay of the no set-UID executables crowd. (grantpt() has to allow an unprivileged user to change the ownership and permissions of a special device file that it does not necessarily own, remember.) So there was still the window of opportunity, and Linux still had to maintain a lock for unlockpt().

Its "new" devpts filesystem (where "new" means introduced quite a few years ago, now) almost permits the same way of doing things as on FreeBSD with devfs, however. There are some differences.

  • There is still a "multiplexor" device.
    • In the older "new" devpts system, this was a ptmx device in a different devtmpfs filesystem, with the devpts filesystem containing only the automatically created/destroyed slave device files. Conventionally the setup was /dev/ptmx and an accompanying devpts mount at /dev/pts.
    • But Linux people wanted to have multiple wholly independent instances of the devpts filesystem, for containers and the like, and it turned out to be quite hard synchronizing the (correct) two filesystems when there were many devtmpfs and devpts filesystems. So in the newer "new" devpts system all of the devices, multiplexor and slave, are in the one filesystem. For backwards compatibility, the default was for the new ptmx node to be inaccessible unless one set a new ptmxmode mount option.
    • In the even newer still "new" devpts the ptmx device file in the devpts filesystem is now the primary multiplexor, and the ptmx in the devtmpfs is either a shim provided by the kernel that tries to mimic a symbolic link, a bind mount, or a plain old actual symbolic link to pts/ptmx.
  • The kernel does not always set up the ownership and permissions as grantpt() should. Setting the wrong mount options, either a gid other than the tty GID or a mode other than 0620, triggers fallback behaviour in the GNU C library. In order to reduce grantpt() to a no-operation in the GNU C library as desired, the kernel must not assign the group of the opening process (i.e. there must be an explicit gid setting), the group assigned must be the tty group, and the mode of newly created slave devices must be exactly 0620.

Not switching on /dev/pts/ptmx by default and the GNU C library not wholly reducing grantpt() to a no-op are both because the kernel and the C library are not maintained in lockstep. Each had to operate with older versions of the other. Linux still had to provide an older /dev/ptmx. The GNU C library still has to fall back to running pt_chown if there's not a new devpts filesystem with the correct mount options in place.

The window of opportunity thus still exists for unlockpt() to guard against on Linux, if the devpts mount options are wrong and the GNU C library consequently has to fall back to actually doing something in grantpt().

Further reading

JdeBP
  • 66,967
  • 12
  • 159
  • 343
  • Great answer, especially because it puts in one place the otherwise quite inaccsible account of the historic develpment. condensed the quesiton's answer is hence that it was /or sometimes still is a security feature. My perception, as I see also portrait in your answer is, that if some other process than the "desired" one has access to the slave part of a pseudo-terminal, than this *could be bad* (hence security feature unlockpt to setup security first). Can you confirm that the "could be bad"-ness of a "rougue slave process" is dependent on what the master does with the data, right?.... – humanityANDpeace Oct 24 '18 at 09:34
  • 1
    ... It seems to me *if* or *if not* a slave process being able to read form /write to the slave side of the pseudo-terminal is only an issue if the master side does something that matters to security on its side, is this kind of correct? thanks! – humanityANDpeace Oct 24 '18 at 09:38