IT managers and analysts are expressing surprise at the amount of time it appears to be taking officials at the City of San Francisco to regain full control of the city's FiberWAN network after a disgruntled network administrator allegedly locked access to it by resetting administrative passwords to its switches and routers.

With few details publicly released on what exactly happened, many are suggesting that the network lockout and the city's response to it point to a failure to implement and manage fundamental security controls.

Terry Childs, a network administrator at San Francisco's Department of Telecommunications and Information Services (DTIS) was arrested on July 13 for allegedly tampering with the network, which carries almost 60% of the city government's traffic. He is also alleged to have planted network devices that enabled illegal remote access to the FiberWAN network.

He was jailed last week on $5 million bond after refusing to divulge the passwords he had used to block access to the network. Child, 43, pleaded not guilty to the charges against him at a hearing in San Francisco Superior Court last week. He is scheduled for a bail hearing tomorrow. If convicted on all charges, Childs faces a maximum of seven years in prison.

As of late Monday, the city's efforts to repair the network still remains a "work in progress" according to Ron Vinson, deputy director of the San Francisco Department of Telecommunications and Information Systems (DTIS). Despite around-the-clock efforts with vendors and staff the city is still only in the process of determining "what level of illegal access or what level of tampering" Childs was responsible for, Vinson said.

According to Vinson, the WAN is operating normally, but the city has not yet regained full administrative of all routers. What remains unclear also is whether IT officials have discovered all of the network devices that Childs is alleged to have illegally installed on the WAN, Vinson said. He did not say what exactly the network devices were or what they did, but claimed that Childs had gone to great lengths to hide them from detection. "We have 60 plus departments that are clients of ours," Vinson said and the task now is to find out if Childs managed to install the devices at any of those departments, he said.

For more enterprise computing news, visit Computerworld. Story copyright Computerworld, Inc.