The Application Is Down and Nobody Here Understands the System

Check these four things in order. Change nothing until you have done all four.

1. Scope. Does the server respond? Do other sites on it work? Is it down for everyone or for some users? This tells you whether you have a server problem, an application problem or a permissions problem.
2. Windows Event Viewer. Open the Application and System logs and filter to errors around the time it stopped. Unhandled .NET exceptions, application pool crashes, service failures and disk warnings all land here.
3. IIS logs. In C:\inetpub\logs\LogFiles by default. Look at the status and sub-status codes for failing requests. A 500 points at the application, a 503 at a stopped application pool, a 401 or 403 at identity or permissions.
4. What changed. Windows Updates installed, a deployment, a password rotation, a certificate renewal, a firewall change. Ask in the last 48 hours, not only today.

Common causes on older Windows-hosted .NET systems

• An expired certificate, on the site itself or on something it calls.
• A full disk, usually from logs or temp files. SQL Server stops writing when its drive fills.
• A service account password that expired or was rotated. The application pool, a Windows service or a scheduled task fails to start.
• A Windows Update that changed TLS defaults or restarted the server, with a service that did not come back.
• A stopped application pool. After repeated crashes IIS disables it through rapid-fail protection.
• An upstream dependency. The database, a file share or a third-party API is unreachable or changed.

Three habits that shorten the outage

Change one thing at a time and write down each change with the time. Copy the relevant event log entries before restarting anything, since a restart can clear the symptom and the evidence together. Restore a backup only when you know what failed, because a restore replaces data and often leaves the cause in place.

If the cause is still unclear after these checks, bring in someone who reads .NET stack traces daily. The event log entries and IIS log lines you collected are what they will ask for first.

The hour after recovery

This hour makes the next outage shorter.

• Write down what failed, how you found it and what fixed it.
• Record every expiry date you now know: certificates, service account passwords, licenses, domain.
• Add an alert for the cause: disk space, certificate expiry, a stopped service.
• List what you had to look for: server names, logins, where the logs are.

That page is the start of a runbook. Where to start with an inherited .NET codebase covers building out the rest.