Details on what happened to the core router. To be honest, I don't exactly know. It just won't initialize all network interfaces and that caused iptables to to be all screwed up.
Out of 5 currently used segments only 3 would come up in random order; very weird indeed.



After several power cycles, the problem persisted. We first thought it was the drive. After replacing it with a backup, the problem persisted.

Finally, after re-sitting the RAM it all went back to normal.

We'll keep a close eye on this bad boy and see what happens in the next few days.

Like always check back here for updates. :P

----

Update on this issue: 12/16/07
The problem was obvious yet frustrating. The router would not detect eth0 at boot time and shift all interfaces up one step. That caused all sorts of packet storms going down the other border routers and messing up the traffic flow.
We finally realized this after several hours and immediately fixed the issue.
Just FYI we've got a second core router setup and ready to take the load if any other issues come up along the way.
Things seem to work fine now.

---

Comments

Comments are not available for this entry.

---