WebSphere Cluster Automation — Startup & Health Checks

IBM Middleware
IBM & Middleware

A WebSphere Application Server cluster with manual startup procedures is a liability. This is the automation model I built to make WAS cluster operations reproducible, observable, and safe for on-call engineers who did not write the original deployment.

Key Patterns & Steps
  • Startup order: Deployment Manager (DMGR) → Node Agents → Application Servers — never skip steps or start in parallel
  • Health check script: wsadmin Jython that queries server state via AdminControl; returns 0 (healthy) or 1 (degraded)
  • Automated startup sequence: each step waits for the previous to reach STARTED state before proceeding — no sleep timers
  • JVM heap monitoring: scrape verbose GC logs; alert before OutOfMemoryError, not after
  • Application deployment pipeline: wsadmin script for install, update, and rollback — no manual Admin Console deployments in production
  • Log aggregation: WAS SystemOut.log and SystemErr.log shipped to central logging before rotation deletes them
Lessons Learned
The most common WAS outage I encountered was caused by a Node Agent that appeared started but had lost its DMGR connection. Health checks must verify connectivity, not just process state.
Technologies
WebSphereWASwsadminJythonAutomationIBMJ2EE
← Back to Architecture Notes