1

Overview

I'm running a Xen server on Debian buster. Latest kernel, latest BIOS version, legacy boot.

Booting into Xen server only shows 8 GB of RAM free while recognizing 16 GB of RAM connected.

Booting into Debian without Xen shows all 16 GB of RAM free.

My Question: Any idea what I can do to force Xen to use all 16 GB?

E.g., I can see that there's an error from xl dmesg saying that 32-bit addressing is being used instead of 64-bit because of some mismatch. Can I force ACPI to use 64-bit?

Logs

From Xen

Booting into Xen shows ACPI: 32/64X FACS address mismatch in FADT - da7ed080/0000000000000000, using 32 and uses only 8 GB of RAM from a total available of 16 GB.

user@server:~/ free -m
free -m
              total        used        free      shared  buff/cache   available
Mem:           7387         379        6635           9         373        6841
Swap:         16291           0       16291

user@server:~/ dmidecode -t17
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x0041, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1600 MT/s
        Manufacturer: Samsung
        Serial Number: 03B0650F7097
        Asset Tag: 02151000
        Part Number: M378B5173QH0-CK0  
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0043, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1333 MT/s
        Manufacturer: Kingston
        Serial Number: 05009D00801C
        Asset Tag: 07194900
        Part Number: 99U5402-034.A00LF 
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0045, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1600 MT/s
        Manufacturer: Samsung
        Serial Number: 0330650F7097
        Asset Tag: 02151000
        Part Number: M378B5173QH0-CK0  
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0047, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1333 MT/s
        Manufacturer: Kingston
        Serial Number: 0500A4008015
        Asset Tag: 07194900
        Part Number: 99U5402-034.A00LF 
        Rank: 1
        Configured Memory Speed: 1333 MT/s

user@server:~/ xl dmesg
(XEN) parameter "placeholder" unknown!
(XEN) Xen version 4.11.4-pre (Debian 4.11.3+24-g14b62ab3e5-1~deb10u1) ([email protected]) (gcc (Debian 8.3.0-6) 8.3.0) debug=n  Wed Jan  8 20:16:51 UTC 2020
(XEN) Bootloader: GRUB 2.02+dfsg1-20
(XEN) Command line: placeholder
(XEN) Xen image load base address: 0xdb000000
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009d800 (usable)
(XEN)  000000000009d800 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000d0ad5000 (usable)
(XEN)  00000000d0ad5000 - 00000000d0adc000 (ACPI NVS)
(XEN)  00000000d0adc000 - 00000000d0f18000 (usable)
(XEN)  00000000d0f18000 - 00000000d13af000 (reserved)
(XEN)  00000000d13af000 - 00000000d7eeb000 (usable)
(XEN)  00000000d7eeb000 - 00000000d8000000 (reserved)
(XEN)  00000000d8000000 - 00000000d8760000 (usable)
(XEN)  00000000d8760000 - 00000000d8800000 (reserved)
(XEN)  00000000d8800000 - 00000000d8fae000 (usable)
(XEN)  00000000d8fae000 - 00000000d9000000 (ACPI data)
(XEN)  00000000d9000000 - 00000000da71c000 (usable)
(XEN)  00000000da71c000 - 00000000da800000 (ACPI NVS)
(XEN)  00000000da800000 - 00000000dbe11000 (usable)
(XEN)  00000000dbe11000 - 00000000dc000000 (reserved)
(XEN)  00000000dd000000 - 00000000df200000 (reserved)
(XEN)  00000000f8000000 - 00000000fc000000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed00000 - 00000000fed04000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed20000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000041ee00000 (usable)
(XEN) ACPI: RSDP 000F0490, 0024 (r2 DELL  )
(XEN) ACPI: XSDT D8FED098, 00AC (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FACP D8FF8B50, 010C (r5 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: DSDT D8FED1D0, B97E (r2 DELL    CBX3          14 INTL 20091112)
(XEN) ACPI: FACS DA7ED080, 0040
(XEN) ACPI: APIC D8FF8C60, 0092 (r3 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FPDT D8FF8CF8, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: SLIC D8FF8D40, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
(XEN) ACPI: LPIT D8FF8EB8, 005C (r1 DELL    CBX3           0 AMI.        5)
(XEN) ACPI: SSDT D8FF8F18, 0539 (r1  PmRef  Cpu0Ist     3000 INTL 20120711)
(XEN) ACPI: SSDT D8FF9458, 0AD8 (r1  PmRef    CpuPm     3000 INTL 20120711)
(XEN) ACPI: SSDT D8FF9F30, 01C7 (r1  PmRef LakeTiny     3000 INTL 20120711)
(XEN) ACPI: HPET D8FFA0F8, 0038 (r1 DELL    CBX3     1072009 AMI.        5)
(XEN) ACPI: SSDT D8FFA130, 036D (r1 SataRe SataTabl     1000 INTL 20120711)
(XEN) ACPI: MCFG D8FFA4A0, 003C (r1 DELL    CBX3     1072009 MSFT       97)
(XEN) ACPI: SSDT D8FFA4E0, 34D6 (r1 SaSsdt  SaSsdt      3000 INTL 20091112)
(XEN) ACPI: ASF! D8FFD9B8, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) ACPI: SSDT D8FFDA60, 0587 (r1 AMITCG     PROC        1 INTL 20120711)
(XEN) ACPI: MSDM D8FFDFE8, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
(XEN) ACPI: DMAR D8FFE040, 00B8 (r1 INTEL      HSW         1 INTL        1)
(XEN) ACPI: TCPA D8FFE0F8, 0032 (r2 APTIO4  NAPAASF        1 MSFT  1000013)
(XEN) System RAM: 16292MB (16683156kB)
(XEN) Domain heap initialised
(XEN) ACPI: 32/64X FACS address mismatch in FADT - da7ed080/0000000000000000, using 32
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster
(XEN) xstate: size: 0x340 and states: 0x7

user@server:~/ xl info
host                   : server
release                : 4.19.0-8-amd64
version                : #1 SMP Debian 4.19.98-1 (2020-01-26)
machine                : x86_64
nr_cpus                : 8
max_cpu_id             : 7
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 2
cpu_mhz                : 3591.702
hw_caps                : bfebfbff:77faf3ff:2c100800:00000021:00000001:000027ab:00000000:00000100
virt_caps              : hvm hvm_directio
total_memory           : 16292
free_memory            : 8278
sharing_freed_memory   : 0
sharing_used_memory    : 0
outstanding_claims     : 0
free_cpus              : 0
xen_major              : 4
xen_minor              : 11
xen_extra              : .4-pre
xen_version            : 4.11.4-pre
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : 
xen_commandline        : placeholder
cc_compiler            : gcc (Debian 8.3.0-6) 8.3.0
cc_compile_by          : pkg-xen-devel
cc_compile_domain      : lists.alioth.debian.org
cc_compile_date        : Wed Jan  8 20:16:51 UTC 2020
build_id               : b6822aa1d8f867753b92985e5cb0e806e520a08c
xend_config_format     : 4

From Debian Buster (without Xen)

Booting into Debian buster without Xen yields the full 16 GB and everything is fine.

user@server:~/ free -m
              total        used        free      shared  buff/cache   available
Mem:          15950         388       15310           8         251       15292
Swap:         16291           0       16291
user@server:~/ dmidecode -t17
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 2.7 present.

Handle 0x0041, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM3
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1600 MT/s
        Manufacturer: Samsung
        Serial Number: 03B0650F7097
        Asset Tag: 02151000
        Part Number: M378B5173QH0-CK0  
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0043, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM1
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1333 MT/s
        Manufacturer: Kingston
        Serial Number: 05009D00801C
        Asset Tag: 07194900
        Part Number: 99U5402-034.A00LF 
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0045, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM4
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1600 MT/s
        Manufacturer: Samsung
        Serial Number: 0330650F7097
        Asset Tag: 02151000
        Part Number: M378B5173QH0-CK0  
        Rank: 1
        Configured Memory Speed: 1333 MT/s

Handle 0x0047, DMI type 17, 34 bytes
Memory Device
        Array Handle: 0x0040
        Error Information Handle: Not Provided
        Total Width: 64 bits
        Data Width: 64 bits
        Size: 4096 MB
        Form Factor: DIMM
        Set: None
        Locator: DIMM2
        Bank Locator: Not Specified
        Type: DDR3
        Type Detail: Synchronous
        Speed: 1333 MT/s
        Manufacturer: Kingston
        Serial Number: 0500A4008015
        Asset Tag: 07194900
        Part Number: 99U5402-034.A00LF 
        Rank: 1
        Configured Memory Speed: 1333 MT/s
Community
  • 1
HBSKan
  • 113
  • 4
  • Thanks for explaining the ACPI error message. I'm checking from Dom0. I've edited the question to add the output of `xl info` (which shows a total memory of 18 GB and only 8 GB free). – HBSKan Feb 12 '20 at 14:58

1 Answers1

3
total_memory           : 16292
free_memory            : 8278

in xl info’s output shows that Xen is using all the available memory, but not in the way you’re expecting: it’s seeing all 16GiB, but has left just over 8GiB unallocated. So your dom0 sees just under 8GiB of memory, and the rest is available for use by domUs.

If you want to allocate more memory to your dom0, you’ll need to edit your boot loader configuration.

(The 32/64X error message doesn’t mean the system is switching to 32-bit addressing; it means there’s an address mismatch in the ACPI tables, and the kernel is using the valid address, which is the 32-bit one here. That only affects how it finds that particular ACPI table.)

Stephen Kitt
  • 411,918
  • 54
  • 1,065
  • 1,164
  • You are brilliant! I think you just solved it. Setting `dom0_mem` in `/etc/default/grub`, disabling `autoballoon` in `xl.conf`, and running `update-grub` is now showing 16G of RAM after reboot. Thank you! – HBSKan Feb 12 '20 at 15:37
  • 1
    You’re welcome! Note that this rather defeats the purpose of running Xen, since you won’t have any memory left to run other domains alongside your dom0. – Stephen Kitt Feb 12 '20 at 15:47
  • 1
    Yep! I was just testing to see if it'll (successfully) let me play around with that number. Down to 1GB we go :D – HBSKan Feb 12 '20 at 15:48