Performing maintenance on GFS2 inside pacemaker cluster

Question

I will need to perform a maintenance on one of storage servers that provides GFS2 volume to a three node pacemaker cluster. Same cluster has an addition of 2 GFS2 volumes as well.

Would it be safe to run pacemaker resource disable on GFS2 resource which needs to be stopped due to maintenance, without risking other GFS2 volumes to be stopped or possibly cluster fenced?

These are constraints:

Ordering Constraints:
  start dlm-clone then start clvmd-clone (kind:Mandatory)
  start clvmd-clone then start gfs2-ISO-clone (kind:Mandatory)
  start clvmd-clone then start gfs2-shared-clone (kind:Mandatory)
  start clvmd-clone then start gfs2-qcow-clone (kind:Mandatory)
Colocation Constraints:
  clvmd-clone with dlm-clone (score:INFINITY)
  gfs2-ISO-clone with clvmd-clone (score:INFINITY)
  gfs2-shared-clone with clvmd-clone (score:INFINITY)
  gfs2-qcow-clone with clvmd-clone (score:INFINITY)

The volume i would like to stop is gfs2-qcow that is gfs2-qcow-clone. If i run pcs resource disable gfs2-qcow-clone will other GFS2 volumes die?

score 0 · Answer 1 · answered Aug 30 '19 at 14:56

In my case we needed to do a maintenance on the RAIDs that we have on our customer, created with mdadm (RAID through software) - we use some RAID resources.

To do this I put the cluster in maintenance with ***pcs property set maintenance-mode=true***, so with this all the fencing resources will be temporary off, to no Stonith event surprised happen. And with this we performed our maintenance. We changed some bad disks and we assemble our RAID and just remove from maintenance-mode and all the resources RAIDs becomed ok.

So, If you don't want your cluster beeing fencing you have two options: maintenance-mode or disable stonith, both with *pcs property*. Pcs property set stonith-enabled=false or pcs property set maintenance-mode=true.

But I recommend you use the maintenance-mode, because this will disable the Pacemaker CRM, and we can do any changes necessary on the resources on Linux or Hardware, and then just came back with the cluster.

On how to disable a specific feature, I did a test in my lab environment to show you what happens if you do pcs resource disable <resource>.

I created a ocf:heartbeat:IPaddr2 named "ip-teste":

sudo pcs resource show ip-teste Resource: ip-teste (class=ocf provider=heartbeat type=IPaddr2) Attributes: cidr_netmask=24 ip=192.168.6.18 Operations: monitor interval=30s (ip-teste-monitor-interval-30s) start interval=0s timeout=20s (ip-teste-start-interval-0s) stop interval=0s timeout=20s (ip-teste-stop-interval-0s)

See:

[leonardo.araujo@sdumont0 ~]$ sudo pcs status

Online: [ sdumont0-ha sdumont1-ha ]

Full list of resources: ip-teste (ocf::heartbeat:IPaddr2): Started sdumont1-ha

Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled

Then I disabled he:

[leonardo.araujo@sdumont0 ~]$ sudo pcs resource disable ip-teste [leonardo.araujo@sdumont0 ~]$ sudo pcs status

9 resources configured (1 DISABLED)

Online: [ sdumont0-ha sdumont1-ha ]

Full list of resources: ip-teste (ocf::heartbeat:IPaddr2): Stopped (disabled)

Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled

Then I enabled he back:

[leonardo.araujo@sdumont0 ~]$ sudo pcs resource enable ip-teste [leonardo.araujo@sdumont0 ~]$ sudo pcs status

2 nodes configured 9 resources configured

Online: [ sdumont0-ha sdumont1-ha ]

Full list of resources: ip-teste (ocf::heartbeat:IPaddr2): Started sdumont1-ha

Daemon Status: corosync: active/disabled pacemaker: active/disabled pcsd: active/enabled

Only that particular feature will be disabled. BUT, if another features depend on this feature, both will be stopped, one because you disabled it and the other because it NEEDs the feature that you have disabled.

I see in your constraints that you only specify the start and location of your resources, I don't see any dependencies on they. So, if is the case and I'm correct, no resource needs the `gfs2-qcow` online, just needs to know where he is and start when he started, you can disabled he and do your maintenance. — Leonardo Magdanello, Aug 30 '19 at 15:07
Thanks a lot ! I will give this a go during the next week and see how it goes. I was thinking about putting cluster in maintenance mode as well. I'll report back either way! Thanks again — Marko Todoric, Sep 01 '19 at 17:43

Performing maintenance on GFS2 inside pacemaker cluster

1 Answers1