Forums Outage - Tuesday, 6am Update

The Inside Carolina Forums went offline again on Sunday and were restored early Tuesday morning. This was not a planned outage.

Tuesday, 6am update: Scout has restored the forums. As a result of corrupted backups, the last 10 days of postings were not able to be recovered. Subscribers will be compensated for the lost time. To get more details, and read notes from both the IC Staff and Scout, click here.

7pm update: No end in sight. Latest word from Scout cited that "repeated attempts to restore the forums database cluster from backups have all failed." They said they are bringing in additional hardware to continue restoration efforts.

3:00pm update: We have no new information beyond the fact that the outage continues and Scout continues to try different processes to bring the forums back online. Since all prior ETAs from Scout that we're relayed have not been realized, we'll refrain from providing any more of them.

9:30am update: Scout just informed us that the situation is "at least two more hours" from being resolved.

Monday, 8am update: The forums were not restored overnight. We are awaiting a morning status update from Scout.

11:30pm update: Still down and still no new ETA of when they'll be back up. The origin of this outage is explained by Scout as a "routine configuration change to fix a slow-performing drive" that then caused various aspects of the system's stressed and obsolete datacenter to fail.

Scout management has provided this network-wide statement: "We apologize for the forums issues this past week. They arise from having to replace obsolete hardware that had not been updated under previous ownership. These updates had originally been planned for after NCAA basketball season, to minimize intrusiveness, but we obviously now will be accelerating this work. We will do whatever it takes to provide stability, speed and capacity, but will err on the side of getting things right for the long haul as opposed to a Band-Aid approach."

11pm update: Repeated attempts to fix the issue and reboot the system have obviously failed, as the forums are still down. Technical details are expected to be provided to us from Scout management shortly. We will pass the important details along as soon as we get them.

8pm update: The forums appeared to show signs of coming back online, but are clearly not operable at this point. We await an update from Scout.

6 pm update: The forums remain offline, and Scout now says they are still 2-3 hours from being fixed. We will continue to update you as we receive more information and we are actively monitoring the situation.

4pm update: Scout says the problem - and its solution - has been identified and the forums will need to be taken offline for about two hours in order to fix it. We have not been given any details about what specifically went wrong and what fix is required.

Sunday, 2pm update: An Inside Carolina staffer reported to us that he was unable to make a post on the forums. We have confirmed this problem and reported it to Scout's tech support team.

We apologize for this inconvenience and thank you for your continued patience while it's being resolved.

- The IC Staff

Inside Carolina Top Stories