You can specify an arbitrary display, but you won’t get far if there’s no corresponding X server. The display number is specified when the X server is started, by whatever starts the X server — typically your display manager, or yourself in your Xpra example. It’s :0 by default (see the Xserver manpage). It can be chosen arbitrarily, but the X server won’t start if the corresponding resources aren’t available (port 6000 + the display number if it’s configured to listen on TCP, /tmp/.X11-unix/X followed by the display number if it’s configured to listen on a Unix domain socket, etc.). The kernel isn’t involved.