Recovery Procedures and System Restoration

Why Recovery Is More Than Just Turning Things Back On

After an incident has been contained and the malicious code or attacker access has been eradicated, the temptation is to restore services as fast as possible. Rushing that step is one of the most common ways organizations experience a second compromise from the same incident. Recovery must be deliberate, verified, and fully documented before any system is returned to production.

Recovery is the fourth phase of the NIST SP 800-61 incident response lifecycle (preparation → detection/analysis → containment/eradication → recovery). Its goal is to return affected systems to a trusted, fully patched, and monitored state while confirming the original attack vector is closed.

Restoring from a Known-Good State

The safest starting point for a compromised system is a verified clean snapshot or offline backup taken before the incident — or, better, from a scheduled pre-incident baseline. Before you restore:

• Confirm the snapshot or backup is clean. Check the creation timestamp against the earliest evidence of compromise in your logs. If the snapshot predates the attacker's foothold, it is a candidate for restoration.
• Verify file integrity. Where your backup solution supports it, check cryptographic hashes of critical files against stored values.
• Isolate the restore target. Keep the system off the production network until all validation steps are complete.

In a virtualized lab environment, rolling back to a clean snapshot is the standard exercise. In production, restoration may involve bare-metal recovery, image deployment, or rebuilding from a hardened gold image, depending on the organization's backup strategy.

Applying Patches and Resetting Credentials

Restoring to a pre-incident state is not enough on its own. You must close the vulnerability or credential weakness the attacker exploited, or you will simply be re-compromised.

Patch management steps during recovery:

1. Identify which vulnerability was exploited (from your analysis phase).

2. Apply the relevant vendor patch or configuration fix immediately after restoration.

3. Apply all other outstanding patches so the system is fully current before it goes back online.

4. Document each patch applied: KB number or CVE reference, date, and technician name.

Credential reset requirements: If the incident involved credential compromise — stolen passwords, pass-the-hash, or service account abuse — every affected account must have its password reset before recovery is complete. This includes:

• Local administrator accounts
• Service accounts used by the compromised application
• Any domain accounts that authenticated on the system during the attack window

Follow your organization's credential-compromise playbook, which will specify minimum password length, complexity, and whether multi-factor authentication enrollment is required before the account is re-enabled.

Post-Recovery Vulnerability Scanning

Once patches are applied and credentials are reset, run a vulnerability scan to confirm the attack surface matches the expected baseline.

Using OpenVAS: OpenVAS (part of the Greenbone Vulnerability Management suite) performs authenticated or unauthenticated scans and produces a report categorized by severity (Critical, High, Medium, Low, Log). After recovery, a full scan of the restored host should show no critical or high findings related to the original attack vector.

Using Nmap scripts (NSE): Nmap's scripting engine includes scripts that check for specific CVEs, weak configurations, and open services. A targeted scan with appropriate NSE scripts can quickly confirm whether a specific vulnerability (for example, an open SMB port with EternalBlue exposure) has been closed.

Comparing to a baseline: Your pre-incident scan output is your reference. Compare the post-recovery scan to that baseline line by line. Any new finding that was not present in the baseline needs to be investigated before the system goes live. Any finding from the baseline that represented the exploited vulnerability must now be absent.

Completing the Recovery Checklist

A recovery checklist is not a formality — it is your legal and operational record that due care was taken. A standard checklist covers:

• System restored from verified clean source ✓
• All outstanding patches applied and documented ✓
• Exploited vulnerability confirmed closed (scan evidence attached) ✓
• All affected credentials reset ✓
• Logging and monitoring re-enabled and verified ✓
• Antivirus/EDR definitions updated ✓
• Functional testing completed (applications start, services respond) ✓
• System owner or supervisor sign-off obtained ✓
• User notification sent (if user data or accounts were affected) ✓
• Lessons-learned notes captured for post-incident report ✓

Every item must be marked complete with a date and the responsible technician's name. Unsigned or undated checklists are not acceptable records.

The Risk of Skipping Validation

Re-infection from the same attack vector is a documented pattern in incident response. Common causes include:

• Restoring a backup that was already compromised
• Forgetting to patch one system in a multi-host environment
• Failing to reset a service account that was harvested by the attacker
• Returning the system to production before monitoring is restored, so the second compromise is not detected

Talking through these risks with a partner during your checklist review is not just a class exercise — it builds the habit of treating each checklist item as a genuine safeguard, not a box to check.

Safety and Professional Notes

In physical data center work, follow all site safety rules: do not power-cycle servers without authorization, confirm proper equipment grounding, and follow your facility's change-management process before touching production hardware. In a lab environment, never restore or reset systems that belong to another student's exercise without explicit instruction from your instructor.