A surprising number of network outages are self-inflicted: a change that seemed simple, a step done out of order, a rollback nobody had planned. The Method of Procedure (MOP) is the service provider industry's answer to that problem. It is a written, step-by-step plan for a change, reviewed before the maintenance window and followed during it.
MOPs are standard practice in carrier networks and increasingly common in enterprise data centers. This article explains what makes a good MOP and provides a template you can adapt.
Why MOPs matter
- They force thinking before doing. Writing the steps exposes gaps, dependencies and risks while there is still time to fix them.
- They make changes reviewable. Peers and change boards can review a document. They can't review what is in an engineer's head.
- They make changes repeatable. A good MOP for a software upgrade can be reused for the next 50 devices.
- They make rollback real. A rollback plan written in advance, with specific commands, is far more reliable than improvising at 2 AM.
- They protect the engineer. Following an approved MOP is strong evidence of due diligence if something goes wrong anyway.
Characteristics of a good MOP
- Specific: exact device names, commands and expected outputs, not "upgrade the software".
- Sequenced: numbered steps in the order they will be executed.
- Verifiable: every major step has a check that proves it worked.
- Reversible: each phase has a defined rollback, and there are clear go/no-go decision points.
- Readable under pressure: someone tired, in the middle of the night, should be able to follow it without interpretation.
MOP template
1. Change summary
- Change ID / ticket:
- Title: for example "Upgrade DC1 leaf switches to version X.Y.Z"
- Author / reviewer / approver:
- Maintenance window: date, start and end time, time zone
- Estimated duration: including rollback time
- Risk level: low / medium / high, with justification
2. Purpose and scope
- Why the change is needed: security fix, capacity, new feature, end of support
- Devices, services and sites in scope, as an explicit list
- What is explicitly out of scope
3. Impact assessment
- Expected service impact: none, degraded or outage, with duration
- Affected customers, applications or business units
- Redundancy relied on during the change, for example "traffic fails over to leaf-08 during leaf-07 reload"
- Notification plan: who is told, when and how
4. Prerequisites
- Software images downloaded and checksums verified
- Configuration backups taken and stored
- Lab testing completed, with results referenced
- Console or out-of-band access confirmed for every device
- Required people and escalation contacts available, including vendor support if needed
- Monitoring maintenance mode scheduled for the devices in scope
5. Pre-change checks
Capture a baseline so you can compare after the change:
- Device health: CPU, memory, environmental status, existing alarms
- Interface status and error counters
- Routing state: neighbor counts, prefix counts, key routes
- Traffic levels on key links
- Service tests: ping and traceroute to key destinations, application checks
Save the outputs. "It looked fine before" is not evidence; a saved command output is.
Go/no-go #1: if the pre-checks show unexpected alarms or degraded redundancy, stop and reschedule.
6. Implementation steps
Number every step. For each one, include:
| # | Action | Command / detail | Expected result | Checked |
|---|---|---|---|---|
| 6.1 | Drain traffic from device | (exact commands) | Traffic moved to peer | ☐ |
| 6.2 | Install software | (exact commands) | Install completes, no errors | ☐ |
| 6.3 | Reload | (exact commands) | Device returns within N minutes | ☐ |
| 6.4 | Verify version | (show command) | Version X.Y.Z | ☐ |
Go/no-go #2 (and onward): add decision points after each high-risk step. Define the conditions that trigger rollback, for example "device not reachable 20 minutes after reload".
7. Post-change verification
Repeat the pre-change checks and compare:
- No new alarms
- Interfaces, routing neighbors and prefix counts match the baseline
- Traffic back on its normal paths
- Service tests pass
- Monitoring shows the device healthy, and maintenance mode ended
8. Rollback plan
- Triggers: the specific conditions under which you roll back
- Steps: numbered and specific, just like implementation
- Time required: make sure it fits inside the window
- Verification after rollback: the same checks as post-change
9. Communication and closure
- Notify stakeholders of completion or rollback
- Update documentation, inventory and diagrams
- Record lessons learned and update the MOP template if something could be improved
MOPs for monitoring platforms
Monitoring and management platforms need MOPs too, and they are often forgotten. When upgrading a fault, performance or orchestration platform, include:
- Visibility during the change: how will you detect network problems while monitoring is down? A secondary collector, a temporary manual watch, or a scheduled window with low risk.
- Data continuity: will polling data or traps be lost during the upgrade? Can collectors buffer?
- Integration checks: ticketing, email, Grafana data sources and APIs all work after the upgrade. Automated API tests (for example a Postman/Newman collection) make this fast.
- Customizations: custom event mappings, scripts and reports survive the upgrade.
Common mistakes
- Writing the MOP after the change, or during it
- Rollback plans that say "restore previous configuration" without the commands
- No baseline captured, so nobody can prove whether something changed
- Scope creep during the window ("while we're here…")
- Skipping peer review because the change is "routine"
Key takeaways
- A MOP is a specific, sequenced, verifiable and reversible plan for a change.
- Capture a baseline before the change and compare after it.
- Build go/no-go decision points and a real rollback plan with exact steps.
- Apply the same discipline to monitoring platform upgrades, and plan for visibility while they are down.