Oracle Linux and ESX hot add cpu and memory

I'm still quite new in Vmware ESX environment and features which I'm using now as my lab (curiosity of VMWare Guru Program).
This time I decided to test hot add CPU and memory functionality. It looks very interesting and in past years hot cpu / memory games where restricted to sophisticated hardware only - now everybody can test is at home.

My lab is basesd on Intel 4 cores and VM are running Oracle Linux 6.3 with or without Oracle Enterprise Kernel. Here are my findings:

1. Hot add CPU / memory is disabled by default and has to be enabled when VM is down. To do so you have to edit your VM configuration and go to options.

2. When feature is enabled you can ADD CPU or memory by editing your VM configuration. Keep in mind that there is no possibility to decrease number of CPU or memory.

3. Here is a table with supported options for ESX 5i and Oracle Linux with or without UEK

I tested adding a CPU to system for RedHat and UEK kernel. As you can find in table linked above adding memory is supported for RedHat kernels only.

RedHat kernel

OEL 6.3 with RedHat kernel running with 1 CPU

[root@oraclese ~]# uname -a
Linux oraclese.localdomain 2.6.32-279.5.2.el6.x86_64 #1 SMP Thu Aug 23 12:05:59 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@oraclese ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc up arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat epb xsaveopt pln pts dts
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

New CPU has been added in vSphere client. Below is output from messages log.

Jan 13 15:17:25 oraclese kernel: SMP alternatives: switching to SMP code
Jan 13 15:17:25 oraclese kernel: Booting Node 0 Processor 1 APIC 0x1
Jan 13 15:17:25 oraclese kernel: Disabled fast string operations
Jan 13 15:17:25 oraclese kernel: mce: CPU supports 0 MCE banks
Jan 13 15:17:25 oraclese kernel: x86 PAT enabled: cpu 1, old 0x0, new 0x7010600070106
Jan 13 15:17:25 oraclese kernel: Skipped synchronization checks as TSC is reliable.
Jan 13 15:17:25 oraclese kernel: microcode: CPU1 sig=0x206a7, pf=0x1, revision=0x14
Jan 13 15:17:25 oraclese kernel: platform microcode: firmware: requesting intel-ucode/06-2a-07
Jan 13 15:17:25 oraclese firmware.sh[2070]: Cannot find  firmware file 'intel-ucode/06-2a-07'
Jan 13 15:17:25 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:17:25 oraclese kdump: failed to stop
Jan 13 15:17:25 oraclese kdump: No crashkernel parameter specified for running kernel

Now system is running with two CPU's

[root@oraclese ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat epb xsaveopt pln pts dts
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc aperfmperf unfair_spinlock pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat epb xsaveopt pln pts dts
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Adding memory - OEL 6.3 wtih RedHat kernel - system running with 1 GB of RAM

[root@oraclese ~]# uname -a
Linux oraclese.localdomain 2.6.32-279.5.2.el6.x86_64 #1 SMP Thu Aug 23 12:05:59 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux
[root@oraclese ~]# free
             total       used       free     shared    buffers     cached
Mem:       1016680     190384     826296          0       4736      77112
-/+ buffers/cache:     108536     908144
Swap:      2097144          0    2097144

Additional 1 GB has been added via vSphere Client. Below is output from messages log.

Jan 13 15:40:54 oraclese kernel: init_memory_mapping: 0000000040000000-0000000048000000
Jan 13 15:40:54 oraclese kernel: Built 1 zonelists in Node order, mobility grouping on.  Total pages: 274190
Jan 13 15:40:54 oraclese kernel: Policy zone: Normal
Jan 13 15:40:54 oraclese kernel: init_memory_mapping: 0000000048000000-0000000050000000
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kernel: init_memory_mapping: 0000000050000000-0000000058000000
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kernel: init_memory_mapping: 0000000058000000-0000000080000000
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: failed to stop
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel
Jan 13 15:40:54 oraclese kdump: No crashkernel parameter specified for running kernel

Now Linux reconized 2 GB of RAM

[root@oraclese ~]# free
             total       used       free     shared    buffers     cached
Mem:       2065256     211840    1853416          0       4764      77328
-/+ buffers/cache:     129748    1935508
Swap:      2097144          0    2097144

Oracle UEK release 2 kernel

OEL 6.3 running with Oracle UEK release 2

[root@oraclese ~]# uname -a
Linux oraclese.localdomain 2.6.32-300.3.1.el6uek.x86_64 #1 SMP Fri Dec 9 18:57:35 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
[root@oraclese ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc up arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

New CPU has been added in vSphere client. Below is output from messages log.

Jan 13 15:38:12 oraclese kernel: processor LNXCPU:01: registered as cooling_device1
Jan 13 15:38:12 oraclese kernel: SMP alternatives: switching to SMP code
Jan 13 15:38:12 oraclese kernel: Booting processor 1 APIC 0x1 ip 0x6000
Jan 13 15:38:12 oraclese kernel: Initializing CPU#1
Jan 13 15:38:12 oraclese kernel: CPU: Physical Processor ID: 2
Jan 13 15:38:12 oraclese kernel: CPU: L1 I cache: 32K, L1 D cache: 32K
Jan 13 15:38:12 oraclese kernel: CPU: L2 cache: 256K
Jan 13 15:38:12 oraclese kernel: CPU: L3 cache: 6144K
Jan 13 15:38:12 oraclese kernel: CPU 1/0x1 -> Node 0
Jan 13 15:38:12 oraclese kernel: mce: CPU supports 0 MCE banks
Jan 13 15:38:12 oraclese kernel: x86 PAT enabled: cpu 1, old 0x0, new 0x7010600070106
Jan 13 15:38:12 oraclese kernel: CPU1: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz stepping 07
Jan 13 15:38:12 oraclese kernel: Skipping synchronization checks as TSC is reliable.
Jan 13 15:38:12 oraclese kernel: microcode: CPU1 sig=0x206a7, pf=0x1, revision=0x14
Jan 13 15:38:12 oraclese kernel: platform microcode: firmware: requesting intel-ucode/06-2a-07
Jan 13 15:38:12 oraclese firmware.sh[2074]: Cannot find  firmware file 'intel-ucode/06-2a-07'
Jan 13 15:38:12 oraclese kdump: kexec: failed to unload kdump kernel
Jan 13 15:38:12 oraclese kdump: failed to stop
Jan 13 15:38:12 oraclese kdump: No crashkernel parameter specified for running kernel

Now system is running with two CPU's

[root@oraclese ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
stepping        : 7
cpu MHz         : 3110.469
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave avx hypervisor lahf_lm ida arat
bogomips        : 6220.93
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

regards, Marcin