Re: 4 node cluster - only two nodes coming up

Sebastian> Well, this is the 1 case, where Load-Sharing Mirrors would
Sebastian> have been helpful...  Too late now, though.

Maybe... it's not clear if it would have helped all that much.  What
we should have done in hindsight, since we had the capacity, would be
to split off two nodes of the cluster ahead of the move, then
snapmirror between them.

It still would have been a pain, but recovery would have been
simpler.  We're still working on issues, but most things are working.

John



Sebastian> Sebastian

Sebastian> On 17.05.2021 02:01, John Stoffel wrote:

Sebastian>     The two nodes came up ok, the problem is the svms with a root ok on the missing pair of nodes.  Escalating with netapp.

Sebastian>     Not sure we have the time to wait for elevator repair to bring the missing pair to the new location, ten hours by truck.  Been a hellosh
Sebastian>     weekend.
   
Sebastian>     I wonder if pod sharing rootvols would have saved us some pain?  Maybe not since updates wouldn’t be possible.  

Sebastian>     Sent from my iPhone

Sebastian>         On May 16, 2021, at 2:18 AM, [email protected] wrote:

Sebastian>         You can only bring up two nodes if one of them has epsilon. Otherwise no configuration changes are possible and that includes
Sebastian>         breaking snap mirror.
       
Sebastian>         If those two nodes do have epsilon, it is just normal procedure to failover LIF. After snap mirror break destinations volumes are not
Sebastian>         renamed - they remain exactly as they are. Nothing special needs to be done when two remaining nodes arrive - they just join cluster
Sebastian>         normally.
       
Sebastian>         If these two nodes do not have epsilon, your best bet is to wait for the remaining cluster nodes to arrive. I am not aware of
Sebastian>         possibility to force epsilon in this case. May be it exists, but you certainly need support case to obtain it and any follow up step.
       
Sebastian>         Скачайте Outlook для iOS
Sebastian>         -------------------------------------------------------------------------------------------------------------------------------------
Sebastian>         От: Пользователь Toasters <[email protected]> от имени пользователя John Stoffel <[email protected]>
Sebastian>         Отправлено: воскресенье, мая 16, 2021 04:54
Sebastian>         Кому: <[email protected]>
Sebastian>         Тема: 4 node cluster - only two nodes coming up
Sebastian>          

Sebastian>         Hi all,
       
Sebastian>         We're in the middle of hell, where our 4-node FAS8060 cluster was
Sebastian>         shutdown cleanly for a move, but only one pair made it onto the truck
Sebastian>         to the new DC.  Luckily I have all the volumes snapmirrored between
Sebastian>         the two pairs of nodes and their aggregates. 
       
Sebastian>         But now I need to bring up the pair that made the trip, figure out
Sebastian>         which mirrors are source and which are destination on this pair, and
Sebastian>         then break the destination ones so I can promote them to read-write.
       
Sebastian>         This is not something I've practiced, and I wonder that if I have
Sebastian>         volume foo, mounted on /foo, and it's snapmirror is volume foo_sm,
Sebastian>         when I do the break, will it automatically mount to /foo?  I guess
Sebastian>         I'll find out later tonight, and I can just unmount and remount. 
       
Sebastian>         I think this is all good with just a simple 'snapmirror break ...' but
Sebastian>         then when we get the chance to rejoin the other two nodes into the
Sebastian>         cluster down the line, I would asusme I just have to (maybe) wipe the
Sebastian>         old nodes and rejoin them one at a time.  Mostly because by that point
Sebastian>         I can't have the original source volumes come up and cause us to lose
Sebastian>         all the writes that have happened on the now writeable destination
Sebastian>         volumes. 
       
Sebastian>         And of course there's the matter of getting epsilon back up and
Sebastian>         working on the two node cluster when I reboot it.  Along with all the
Sebastian>         LIFs, etc.  Not going to be a fun time.  Not at all...
       
Sebastian>         And of course we're out of support with Netapp.  Sigh...
       
Sebastian>         And who knows if the pair that came down won't lose some disks and end
Sebastian>         up losing one or more aggregates as well.  Stressful times for sure.
       
Sebastian>         So I'm just venting here, but any suggestions or tricks would be
Sebastian>         helpful.
       
Sebastian>         And of course I'm not sure if the cluster switches made it down here
Sebastian>         yet.
       
Sebastian>         Never put your DC on the second floor if there isn't a second freight
Sebastian>         elevator.  Or elevator in general.  Sigh...
       
Sebastian>         John
Sebastian>         _______________________________________________
Sebastian>         Toasters mailing list
Sebastian>         [email protected]
Sebastian>         https://www.teaparty.net/mailman/listinfo/toasters

Sebastian>     _______________________________________________
Sebastian>     Toasters mailing list
Sebastian>     [email protected]
Sebastian>     https://www.teaparty.net/mailman/listinfo/toasters


_______________________________________________
Toasters mailing list
[email protected]
https://www.teaparty.net/mailman/listinfo/toasters