1

On my computer, I have 2 discrete GPUs. I've been using VFIO to pass the second GPU to a Windows VM to work with some programs. Now I want to pass my more powerful first GPU to the Windows VM in order to play some games.(I can create a second Windows VM if necessary, this is not a concern).

I've checked with a script if the first GPU has its own IOMMU group, and it has.

The problem is, Gnome DE and gdm3 starts without an issue when I boot normally or pass the second GPU, but it doesn't when I pass the first GPU.

The monitors do turn on/off depending on the passed GPU.

I've tried to restart gdm3, kill gnome-shell, reboot, but it doesn't seem to have any effect getting to the DE. GDM3 started somehow, but it just got the password and returned to the user selection menu.

The output of the script: (excluding non-necessary stuff, just the 2 GPUs)

IOMMU Group 12:
    00:15.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] SB700/SB800/SB900 PCI to PCI bridge (PCIE port 0) [1002:43a0]
    05:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series] [1002:68f9]
    05:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series] [1002:aa68]
IOMMU Group 14:
    01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin [Radeon RX 460/560D / Pro 450/455/460/555/555X/560/560X] [1002:67ef] (rev cf)
    01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1002:aae0]

journalctl -u gdm when it does start (No passthrough):

Feb 11 17:29:53 Alienus-PC systemd[1]: Starting GNOME Display Manager...
Feb 11 17:29:54 Alienus-PC systemd[1]: Started GNOME Display Manager.
Feb 11 17:29:57 Alienus-PC gdm-autologin][1490]: gkr-pam: no password is available for user
Feb 11 17:30:00 Alienus-PC gdm-autologin][1490]: pam_unix(gdm-autologin:session): session opened for user alienus by (uid=0)

journalctl -u gdm when it does not start (First GPU passthough):

Feb 11 17:25:58 Alienus-PC systemd[1]: Starting GNOME Display Manager...
Feb 11 17:25:58 Alienus-PC systemd[1]: Started GNOME Display Manager.
Feb 11 17:25:58 Alienus-PC gdm-autologin][2650]: gkr-pam: no password is available for user
Feb 11 17:25:58 Alienus-PC gdm-autologin][2650]: pam_unix(gdm-autologin:session): session opened for user alienus by (uid=0)
Feb 11 17:25:58 Alienus-PC gdm-autologin][2650]: gkr-pam: couldn't unlock the login keyring.
Feb 11 17:25:59 Alienus-PC gdm-autologin][2650]: pam_unix(gdm-autologin:session): session closed for user alienus
Feb 11 17:25:59 Alienus-PC gdm3[1359]: GdmDisplay: Session never registered, failing
Feb 11 17:25:59 Alienus-PC gdm-launch-environment][2659]: pam_unix(gdm-launch-environment:session): session opened for user gdm by (uid=0)
Feb 11 17:25:59 Alienus-PC gdm-launch-environment][2659]: pam_unix(gdm-launch-environment:session): session closed for user gdm
Feb 11 17:25:59 Alienus-PC gdm3[1359]: Child process -2688 was already dead.

journalctl -u gdm with debug enabled (/etc/gdm3/custom.conf, First GPU passthrough): https://paste.ubuntu.com/p/cSsDpBynyM/ (the output is about 52k characters, I can't post it here)

Specs of the system:

Emre Talha
  • 155
  • 6
  • Maybe setting the [`DRI_PRIME`](https://docs.mesa3d.org/envvars.html) environment variable will cause things to use a GPU other than the first one. Don't know. Worth trying. – user253751 Feb 11 '23 at 14:51
  • @user253751 where should i add that option? Inside the grub option that I added the `vfio-pci.ids=` part? Inside `.bashrc`? – Emre Talha Feb 11 '23 at 15:06
  • If systemd is what's running gdm then environment variables can be added in the unit file e.g. `Environment=DRI_PRIME=something`. But first I would find out what Mesa (Linux 3D graphics system) thinks the graphics cards are, in both cases. The DRI_PRIME idea is based on the idea that Mesa is trying to use the wrong graphics card. – user253751 Feb 11 '23 at 15:11
  • @user253751 there was no gdm unit file. Instead, there was a `display-manager.service` unit, adding "Environment=DRI_PRIME=1" or `1002:68f9` instead of 1 did not work. I've enabled the debug option in `/etc/gdm3/custom.conf` and i'm inspecting if there are any errors. – Emre Talha Feb 11 '23 at 16:30

1 Answers1

0

This answer solved my problem with not being able to start Gnome with the second GPU.

In order to get into a state with no active X session, I've booted with the first GPU disabled. After that, I switched to tty3 and run Xorg -configure. It created a config file with 21 cards and screens (weird, maybe it was for all the PCI IDs).

After deleting all the other cards except my second GPU, all screens but the first 3, I've copied the config file into /etc/X11/xorg.conf and rebooted.

At first, I wanted to see if this worked when I did not disable the first GPU and booted normally. The numbers of the monitors have changed and when I checked with xrandr --listproviders the provider with the 0 ID was my second GPU. After that, I've rebooted and disabled the first GPU, and it started the Gnome DE without a problem.

My xorg.conf file:

Section "ServerLayout"
    Identifier     "X.org Configured"
    Screen      0  "Screen0" 0 0
    Screen      1  "Screen1" RightOf "Screen0"
    Screen      2  "Screen2" RightOf "Screen1"
    InputDevice    "Mouse0" "CorePointer"
    InputDevice    "Keyboard0" "CoreKeyboard"
EndSection

Section "Files"
    ModulePath   "/usr/lib/xorg/modules"
    FontPath     "/usr/share/fonts/X11/misc"
    FontPath     "/usr/share/fonts/X11/cyrillic"
    FontPath     "/usr/share/fonts/X11/100dpi/:unscaled"
    FontPath     "/usr/share/fonts/X11/75dpi/:unscaled"
    FontPath     "/usr/share/fonts/X11/Type1"
    FontPath     "/usr/share/fonts/X11/100dpi"
    FontPath     "/usr/share/fonts/X11/75dpi"
    FontPath     "built-ins"
EndSection

Section "Module"
    Load  "glx"
EndSection

Section "InputDevice"
    Identifier  "Keyboard0"
    Driver      "kbd"
EndSection

Section "InputDevice"
    Identifier  "Mouse0"
    Driver      "mouse"
    Option      "Protocol" "auto"
    Option      "Device" "/dev/input/mice"
    Option      "ZAxisMapping" "4 5 6 7"
EndSection

Section "Monitor"
    Identifier   "Monitor0"
    VendorName   "Monitor Vendor"
    ModelName    "Monitor Model"
EndSection

Section "Monitor"
    Identifier   "Monitor1"
    VendorName   "Monitor Vendor"
    ModelName    "Monitor Model"
EndSection

Section "Monitor"
    Identifier   "Monitor2"
    VendorName   "Monitor Vendor"
    ModelName    "Monitor Model"
EndSection

Section "Device"
        ### Available Driver options are:-
        ### Values: <i>: integer, <f>: float, <bool>: "True"/"False",
        ### <string>: "String", <freq>: "<f> Hz/kHz/MHz",
        ### <percent>: "<f>%"
        ### [arg]: arg optional
        #Option     "Accel"                 # [<bool>]
        #Option     "SWcursor"              # [<bool>]
        #Option     "EnablePageFlip"        # [<bool>]
        #Option     "SubPixelOrder"         # [<str>]
        #Option     "ZaphodHeads"           # <str>
        #Option     "AccelMethod"           # <str>
        #Option     "DRI3"                  # [<bool>]
        #Option     "DRI"                   # <i>
        #Option     "ShadowPrimary"         # [<bool>]
        #Option     "TearFree"              # [<bool>]
        #Option     "DeleteUnusedDP12Displays"  # [<bool>]
        #Option     "VariableRefresh"       # [<bool>]
    Identifier  "Card0"
    Driver      "amdgpu"
    BusID       "PCI:5:0:0"
EndSection

Section "Screen"
    Identifier "Screen0"
    Device     "Card0"
    Monitor    "Monitor0"
    SubSection "Display"
        Viewport   0 0
        Depth     1
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     24
    EndSubSection
EndSection

Section "Screen"
    Identifier "Screen1"
    Device     "Card1"
    Monitor    "Monitor1"
    SubSection "Display"
        Viewport   0 0
        Depth     1
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     24
    EndSubSection
EndSection

Section "Screen"
    Identifier "Screen2"
    Device     "Card2"
    Monitor    "Monitor2"
    SubSection "Display"
        Viewport   0 0
        Depth     1
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     4
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     8
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     15
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     16
    EndSubSection
    SubSection "Display"
        Viewport   0 0
        Depth     24
    EndSubSection
EndSection
Emre Talha
  • 155
  • 6