Re: 4 node cluster - only two nodes coming up

Hi Tim,
Thanks for the offer to help.  We will just have to see how long it will take to reunite our cluster pairs, though I’d be happy to just dump the missing pair and handle the data loss.

Which won’t be much if anything at all.

Sent from my iPhone

> On May 15, 2021, at 7:04 PM, Timothy Naple <[email protected]> wrote:
> 
> 
> John,
> 
> If it were me, I would take it one step at a time and first get the systems physically in place, cabled up and clusters switches up and running and once you can login on console or SSH then start looking at the aggregates, LIF's, cluster epsilon/quorum and which volumes are source, etc.  Keep me posted if you hit any major roadblocks and I can try to help out.
> 
> Thank you,
> Tim
> 
> From: Toasters <[email protected]> on behalf of John Stoffel <[email protected]>
> Sent: Saturday, May 15, 2021 6:47 PM
> To: <[email protected]> <[email protected]>
> Subject: 4 node cluster - only two nodes coming up
>  
> 
> 
> Hi all,
> 
> We're in the middle of hell, where our 4-node FAS8060 cluster was
> shutdown cleanly for a move, but only one pair made it onto the truck
> to the new DC.  Luckily I have all the volumes snapmirrored between
> the two pairs of nodes and their aggregates.  
> 
> But now I need to bring up the pair that made the trip, figure out
> which mirrors are source and which are destination on this pair, and
> then break the destination ones so I can promote them to read-write.
> 
> This is not something I've practiced, and I wonder that if I have
> volume foo, mounted on /foo, and it's snapmirror is volume foo_sm,
> when I do the break, will it automatically mount to /foo?  I guess
> I'll find out later tonight, and I can just unmount and remount.  
> 
> I think this is all good with just a simple 'snapmirror break ...' but
> then when we get the chance to rejoin the other two nodes into the
> cluster down the line, I would asusme I just have to (maybe) wipe the
> old nodes and rejoin them one at a time.  Mostly because by that point
> I can't have the original source volumes come up and cause us to lose
> all the writes that have happened on the now writeable destination
> volumes.  
> 
> And of course there's the matter of getting epsilon back up and
> working on the two node cluster when I reboot it.  Along with all the
> LIFs, etc.  Not going to be a fun time.  Not at all...
> 
> And of course we're out of support with Netapp.  Sigh...
> 
> And who knows if the pair that came down won't lose some disks and end
> up losing one or more aggregates as well.  Stressful times for sure.
> 
> So I'm just venting here, but any suggestions or tricks would be
> helpful.
> 
> And of course I'm not sure if the cluster switches made it down here
> yet.
> 
> Never put your DC on the second floor if there isn't a second freight
> elevator.  Or elevator in general.  Sigh...
> 
> John
> _______________________________________________
> Toasters mailing list
> [email protected]
> https://www.teaparty.net/mailman/listinfo/toasters

_______________________________________________
Toasters mailing list
[email protected]
https://www.teaparty.net/mailman/listinfo/toasters