Blog

Writing Detailed MOPs, and the Distinction between Planning and - ECG

Written by Mark Lindsey | Nov 26, 2008 5:00:00 AM

Service Providers often want a detailed Method of Procedure (MOP) for any change in their network. Some service providers, like Level(3),
have Engineering people plan the procedure, while Operations people actually do the procedure.

This does encourage careful planning. But sometimes things go wrong;
most MOPs have a back-out procedure, so that any changes can be reversed.

But wouldn't it be better if the change could be made in a planned way, but minor problems be resolved when doing the change? When a
surgeon cuts you ope, he has a plan. But if he gets inside and discovers complications, he'll often try to actually fix the problem.
"There were complications, so the surgery took longer."

Let's step back from big organizations that have all these people and
procedures. In smaller, looser, MOP-free organizations, smart people
do the changes. They often login -- and figure out precisely what
needs to be done while they're logged in. If they run into
complications, then they fix the problems then and there. There's
something attractive about just getting the problem solved.

But then the smart people get burned: sometimes mistakes happen and
create outages. Or, the change appears to work fine, but then the
problems erupt later because of the change. So one force at work is
that the stop-talking-about-it-and-fix-it approach sometimes leads to
outages. The technicians implement change control; for example, they
only make their changes at night.

But another force is this: what if the technicians break things, but
the managers have to take the calls from angry users? The managers
start to implement change control. Another form of change control is
that they want to approve every change, because they don't want to
suffer for the outage caused by the technicians. It seems this kind of
control sometimes prevents problems from being fixed in the
maintenance window.

Change Control systems seem to have these dimensions:

-- When (what times and dates) changes are made

-- Who plans the changes to be made

-- Who approves the changes that are made

-- How the changes are recorded or logged

-- How the changes are tested to ensure that they worked properly.

-- Degree of latitude the changers have to work around complications