Home / LDOM / LDOM – Memory / CPU Reallocation on Hard Partitioning

LDOM – Memory / CPU Reallocation on Hard Partitioning

In LDOM , we can’t dynamically remove/add memory resource when the resources are physically bound. This will make hard-time for administrator and application owners where we need to bring down the guest domain every-time when you re-reconfigure memory resources. Normally Oracle Super-cluster servers are pre-configured with Hard Partitioning using the LDOM virtualization.  In typical LDOM method  , CPU threads are allocated to the guest domains where as in hard partitioning method , whole CPU cores needs to be allocated to the systems. According to the oracle notes ,

  • You cannot use dynamic reconfiguration (DR) to move memory or core resources between running domains when you set the mblock or cid property.

 

You will get below errors when you tried to dynamically allocate the resources using vcpu command. CPU whole-core can be add/remove dynamically when the domain is in bound/active state. But you have to use “add-core” or “remove-core” instead of using “add-vcpu” or “remove-vcpu” .

# ldm remove-vcpu 1 app1node1

Domain app1node1 uses physically bound core resources #

 

1. To Check Whether a Domain is Configured With CPU Whole Cores(Hard Partitioning) and a CPU Cap, Use the below mentioned command.

# ldm list -o resmgmt app1node1
NAME
app1node1

CONSTRAINT
    cpu=whole-core
    max-cores=16
    threading=max-throughput
    physical-bindings=core,memory
#

Verify that the whole-core constraint appears in the output and that the max-cores keyword specifies the maximum number of CPU cores configured for the domain. This shows that system  is using “CPU Whole Cores(Hard Partitioning)” .

 

2.  To get the allocated core details for the specific domain.

# ldm list -o core app1node1
NAME
app1node1

CORE
    CID    CPUSET
    16     (128, 129, 130, 131, 132, 133, 134, 135)
    17     (136, 137, 138, 139, 140, 141, 142, 143)
    18     (144, 145, 146, 147, 148, 149, 150, 151)
    19     (152, 153, 154, 155, 156, 157, 158, 159)
    20     (160, 161, 162, 163, 164, 165, 166, 167)
    21     (168, 169, 170, 171, 172, 173, 174, 175)
    22     (176, 177, 178, 179, 180, 181, 182, 183)
    23     (184, 185, 186, 187, 188, 189, 190, 191)
    24     (192, 193, 194, 195, 196, 197, 198, 199)
    25     (200, 201, 202, 203, 204, 205, 206, 207)
    26     (208, 209, 210, 211, 212, 213, 214, 215)
    27     (216, 217, 218, 219, 220, 221, 222, 223)
    28     (224, 225, 226, 227, 228, 229, 230, 231)
    29     (232, 233, 234, 235, 236, 237, 238, 239)
    30     (240, 241, 242, 243, 244, 245, 246, 247)
    31     (248, 249, 250, 251, 252, 253, 254, 255)

#
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        active     -n----  5001    128   512G     1.9%  1.5%  7h 58m
#

 

3. To dynamically add the CPU cores to the existing active domain, use the below command.

# ldm add-core 2 app1node1
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        active     -n----  5001    144   512G     1.9%  1.5%  7h 58m
#

4. Using the remove-core command , you can reduce the CPU whole core from specific domain.

# ldm remove-core 2 app1node1
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        active     -n----  5001    128   512G     1.9%  1.5%  7h 58m
#

5. You can also set the total number of CPU cores to specific domain.

# ldm set-core 18 app1node1
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        active     -n----  5001    144   512G     1.9%  1.5%  7h 58m
#

Note: Each CPU core has 8 threads in this CPU model.

 

Let’s move to memory part.

When the memory resources are physically bind to the guest domain, you can’t dynamically add/remove from the domains.  You will get below error when you try to do that.

# ldm remove-mem 1g app1node1

Domain app1node1 uses physically bound memory resources Resource removal failed. #

 

1. The below command  shows the memory resources are physically pinned to the specific domain.

# ldm list -o resmgmt app1node1
NAME
app1node1

CONSTRAINT
    cpu=whole-core
    max-cores=16
    threading=max-throughput
    physical-bindings=core,memory
#

 

2. To know extract the detail information  about the physically bound memory resource, use the below command.

# ldm list-constraints app1node1
MEMORY
    SIZE:    512G
    PHYSICAL-BINDINGS
        PA               PSIZE
        0x400000000      8G
        0x600000000      8G
        0xc00000000      8G
        0xe00000000      8G
        0x1400000000     8G
        0x1600000000     8G
        0x1c00000000     8G
        0x1e00000000     8G
        0x9400000000     8G
        0x9600000000     8G
        0x9c00000000     8G
        0x9e00000000     8G
        0x3400000000     8G
        0x3600000000     8G
        0x3c00000000     8G
        0x3e00000000     8G
        0x4400000000     8G
        0x4600000000     8G
        0x4c00000000     8G
        0x4e00000000     8G
        0x5400000000     8G
        0x5600000000     8G
        0x5c00000000     8G
        0x5e00000000     8G
        0x6400000000     8G
        0x6600000000     8G
        0x6c00000000     8G
        0x6e00000000     8G
        0x7400000000     8G
        0x7600000000     8G

 

3. Let’s plan to remove some of the allocated memory resources.  We will bring the total memory size from 512G  to 480G .

 

4. Halt the guest domain in which you are planning to reduce the memory.

 

5. Once the guest domain is stopped, un-bind it .

# ldm unbind  app1node1

 

6. The existing memory allocation is number of 8G memory chunks. Let me remove 4 memory module from this guest domain.

# ldm remove-mem mblock=0x7600000000:8G,0x7400000000:8G,0x6e00000000:8G,0x6c00000000:8G app1node1

 

7. Verify the guest domain memory size. Here you can see that physical memory size has been reduced from 512G to 480G.

# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        inactive   -_----  5001    144   480G     
#

 

8. If you would like to increase the physical memory , you can add it back to the guest domain. If you don;t the PA , you can see the “# ldm list-devices -a memory” to find the free PA.

# ldm add-mem mblock=0x7600000000:8G,0x7400000000:8G,0x6e00000000:8G,0x6c00000000:8G app1node1
# ldm list
NAME             STATE      FLAGS   CONS    VCPU  MEMORY   UTIL  NORM  UPTIME
primary          active     -n-cv-  UART    112   512G      14%   14%  7h 51m
app1node1        inactive   -_----  5001    144   512G     
#

9. Bind the guest domain & start it

# ldm bind app1node1
# ldm start app1node1

 

The mblock property should be used only by an administrator who is knowledgeable about the topology of the system to be configured. This advanced configuration feature enforces specific allocation rules and might affect the overall performance of the system.

 

Hope this article is informative to you.

VMTURBO-CLOUD-CAPACITY

Leave a Reply

Your email address will not be published. Required fields are marked *