In LDOM , we can’t dynamically remove/add memory resource when the resources are physically bound. This will make hard-time for administrator and application owners where we need to bring down the guest domain every-time when you re-reconfigure memory resources. Normally Oracle Super-cluster servers are pre-configured with Hard Partitioning using the LDOM virtualization. In typical LDOM method , CPU threads are allocated to the guest domains where as in hard partitioning method , whole CPU cores needs to be allocated to the systems. According to the oracle notes ,
- [highlight color=”yellow”]You cannot use dynamic reconfiguration (DR) to move memory or core resources between running domains when you set the mblock or cid property.[/highlight]
You will get below errors when you tried to dynamically allocate the resources using vcpu command. CPU whole-core can be add/remove dynamically when the domain is in bound/active state. But you have to use “add-core” or “remove-core” instead of using “add-vcpu” or “remove-vcpu” .
# ldm remove-vcpu 1 app1node1
[box type=”error” align=”” class=”” width=””] Domain app1node1 uses physically bound core resources #[/box]
1. To Check Whether a Domain is Configured With CPU Whole Cores(Hard Partitioning) and a CPU Cap, Use the below mentioned command.
# ldm list -o resmgmt app1node1
NAME
app1node1
CONSTRAINT
cpu=whole-core
max-cores=16
threading=max-throughput
physical-bindings=core,memory
#
Verify that the whole-core constraint appears in the output and that the max-cores keyword specifies the maximum number of CPU cores configured for the domain. This shows that system is using “CPU Whole Cores(Hard Partitioning)” .
2. To get the allocated core details for the specific domain.
# ldm list -o core app1node1
NAME
app1node1
CORE
CID CPUSET
16 (128, 129, 130, 131, 132, 133, 134, 135)
17 (136, 137, 138, 139, 140, 141, 142, 143)
18 (144, 145, 146, 147, 148, 149, 150, 151)
19 (152, 153, 154, 155, 156, 157, 158, 159)
20 (160, 161, 162, 163, 164, 165, 166, 167)
21 (168, 169, 170, 171, 172, 173, 174, 175)
22 (176, 177, 178, 179, 180, 181, 182, 183)
23 (184, 185, 186, 187, 188, 189, 190, 191)
24 (192, 193, 194, 195, 196, 197, 198, 199)
25 (200, 201, 202, 203, 204, 205, 206, 207)
26 (208, 209, 210, 211, 212, 213, 214, 215)
27 (216, 217, 218, 219, 220, 221, 222, 223)
28 (224, 225, 226, 227, 228, 229, 230, 231)
29 (232, 233, 234, 235, 236, 237, 238, 239)
30 (240, 241, 242, 243, 244, 245, 246, 247)
31 (248, 249, 250, 251, 252, 253, 254, 255)
#
# ldm list
NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME
primary active -n-cv- UART 112 512G 14% 14% 7h 51m
app1node1 active -n---- 5001 128 512G 1.9% 1.5% 7h 58m
#
3. To dynamically add the CPU cores to the existing active domain, use the below command.
# ldm add-core 2 app1node1 # ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME primary active -n-cv- UART 112 512G 14% 14% 7h 51m app1node1 active -n---- 5001 144 512G 1.9% 1.5% 7h 58m #
4. Using the remove-core command , you can reduce the CPU whole core from specific domain.
# ldm remove-core 2 app1node1 # ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME primary active -n-cv- UART 112 512G 14% 14% 7h 51m app1node1 active -n---- 5001 128 512G 1.9% 1.5% 7h 58m #
5. You can also set the total number of CPU cores to specific domain.
# ldm set-core 18 app1node1 # ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME primary active -n-cv- UART 112 512G 14% 14% 7h 51m app1node1 active -n---- 5001 144 512G 1.9% 1.5% 7h 58m #
Note: Each CPU core has 8 threads in this CPU model.
Let’s move to memory part.
When the memory resources are physically bind to the guest domain, you can’t dynamically add/remove from the domains. You will get below error when you try to do that.
# ldm remove-mem 1g app1node1
[box type=”error” align=”” class=”” width=””] Domain app1node1 uses physically bound memory resources Resource removal failed. #[/box]
1. The below command shows the memory resources are physically pinned to the specific domain.
# ldm list -o resmgmt app1node1
NAME
app1node1
CONSTRAINT
cpu=whole-core
max-cores=16
threading=max-throughput
physical-bindings=core,memory
#
2. To know extract the detail information about the physically bound memory resource, use the below command.
# ldm list-constraints app1node1
MEMORY
SIZE: 512G
PHYSICAL-BINDINGS
PA PSIZE
0x400000000 8G
0x600000000 8G
0xc00000000 8G
0xe00000000 8G
0x1400000000 8G
0x1600000000 8G
0x1c00000000 8G
0x1e00000000 8G
0x9400000000 8G
0x9600000000 8G
0x9c00000000 8G
0x9e00000000 8G
0x3400000000 8G
0x3600000000 8G
0x3c00000000 8G
0x3e00000000 8G
0x4400000000 8G
0x4600000000 8G
0x4c00000000 8G
0x4e00000000 8G
0x5400000000 8G
0x5600000000 8G
0x5c00000000 8G
0x5e00000000 8G
0x6400000000 8G
0x6600000000 8G
0x6c00000000 8G
0x6e00000000 8G
0x7400000000 8G
0x7600000000 8G
3. Let’s plan to remove some of the allocated memory resources. We will bring the total memory size from 512G to 480G .
4. Halt the guest domain in which you are planning to reduce the memory.
5. Once the guest domain is stopped, un-bind it .
# ldm unbind app1node1
6. The existing memory allocation is number of 8G memory chunks. Let me remove 4 memory module from this guest domain.
# ldm remove-mem mblock=0x7600000000:8G,0x7400000000:8G,0x6e00000000:8G,0x6c00000000:8G app1node1
7. Verify the guest domain memory size. Here you can see that physical memory size has been reduced from 512G to 480G.
# ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME primary active -n-cv- UART 112 512G 14% 14% 7h 51m app1node1 inactive -_---- 5001 144 480G #
8. If you would like to increase the physical memory , you can add it back to the guest domain. If you don;t the PA , you can see the “# ldm list-devices -a memory” to find the free PA.
# ldm add-mem mblock=0x7600000000:8G,0x7400000000:8G,0x6e00000000:8G,0x6c00000000:8G app1node1 # ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL NORM UPTIME primary active -n-cv- UART 112 512G 14% 14% 7h 51m app1node1 inactive -_---- 5001 144 512G #
9. Bind the guest domain & start it
# ldm bind app1node1 # ldm start app1node1
[box type=”warning” align=”” class=”” width=””]The mblock property should be used only by an administrator who is knowledgeable about the topology of the system to be configured. This advanced configuration feature enforces specific allocation rules and might affect the overall performance of the system.[/box]
Hope this article is informative to you.
Leave a Reply