THE EFFECTIVENESS OF ADMINISTRATIVE CONTROLS OVER AN OUTSOURCED CONTRACT

Other format: PDF (271 KB) (Download Accessible PDF Plug-in)

Board of Governors of the Federal Reserve System

REPORT ON THE EFFECTIVENESS OF ADMINISTRATIVE CONTROLS OVER AN OUTSOURCED CONTRACT

OFFICE OF INSPECTOR GENERAL

	BOARD OF GOVERNORS OF THE FEDERAL RESERVE SYSTEM WASHINGTON, D. C.  20551
		OFFICE OF INSPECTOR GENERAL
	June 2, 2004

Mr. Stephen R. Malphrus
Staff Director for Management
Board of Governors of the Federal Reserve System
Washington, DC 20551

Dear Mr. Malphrus:

The Office of Inspector General (OIG) of the Board of Governors of the Federal Reserve System (Board) is pleased to present its Report on the Effectiveness of Administrative Controls over an Outsourced Contract (A0207-A). The OIG recently completed an audit of the Board's outsourcing operations. ¹ As we noted in that report, the audit included a detailed review of fifteen judgmentally-selected contracts providing some level of outsourced services. One of the contracts we reviewed was for the acquisition of information security control reviews as part of the Board's implementation of the Government Information Security Reform Act (GISRA).

Because our review of the security control review contract identified substantial increases in contract costs, as well as potential weaknesses in the contracting and contract-modification processes, we decided to perform a more in-depth evaluation of the effectiveness of administrative controls governing that acquisition. To accomplish our objective, we interviewed management and staff in the Management Division (MGT), the Legal Division (Legal), and the Division of Information Technology (IT), as well as vendor staff who had a working knowledge of the contract. We also analyzed contract documentation, including documentation produced by the vendor. We conducted our fieldwork between July and October 2003 in accordance with generally accepted government auditing standards.

BACKGROUND

In May 2002, the Board executed a sole-source contract with a well-known audit firm to perform information security control reviews; the same firm had been competitively selected in 2001 to perform two control reviews for the Board. The 2002 contract, initially budgeted at $60,000, did not identify the exact systems to be reviewed or the time required to perform each review, although the sole-source justification estimated that six to eight reviews would be conducted. The Board's procurement manager was the contracting officer; the current director of IT (who served as the point of contact for many GISRA-related issues) initiated the acquisition and also served as the contracting officer's technical representative (COTR). The vendor ultimately performed six reviews for a total cost of $573,491, nearly ten times the original budget estimate. Figure 1 provides a timeline for this acquisition and shows the dates that the vendor performed work; the dates that the vendor submitted invoices; and the dates that the Board processed the associated contract modifications, purchase requisitions, and purchase orders.

OBSERVATIONS

Although the director of IT was satisfied that the vendor's work met the Board's requirements, we found that the administrative controls over the acquisition and contract management processes were not implemented in a way that ensured that the Board obtained best value for this acquisition. Specifically, we found that the acquisition was governed by a contract that provided little cost or performance discipline, was not implemented and administered consistent with the Board's Acquisition Policy, and made ineffective use of the GSA schedule. At the heart of the issue was the decision to implement a contract without specifying the expected number of reviews or including a "not-to-exceed" dollar amount. Board officials believed that better management of the quality and cost of the vendor's work could be achieved by incrementally adding requirements for other reviews and related funding as satisfactory work was completed on each review. However, we believe that the potential benefits of implementing such an incremental approach were largely negated because the contract did not provide adequate controls on the vendor over pricing or performance.

Our outsourcing report contains a recommendation to improve statements of work for outsourced contracts, and implementation of this recommendation should address several of the issues discussed above. Specifically, clearly establishing vendor performance expectations (total number of systems, the period of performance, and expected hours to complete each review) would have provided a mechanism to better control costs and evaluate vendor performance. We noted, for example, a significant variance between actual and anticipated costs for each review performed by the vendor. The vendor's estimate per system (although not incorporated into the contract pricing) ranged from $16,000 to $32,000, depending on the systems' size and complexity. Actual costs, however, for five of the systems far exceeded the vendor's highest estimate. Based on the vendor's time reports, we estimated that the 2002 actual cost per system ranged from an average of about $19,000 for small systems to an average of about $71,000 for large systems. ²  We also found that the contract did not include a definitive period of performance. The performance period described in the contract ran, ". . . from the contract execution date through the completion of the project." The vendor's proposal, which is incorporated into the contract, stated that the vendor would develop an appropriate work plan to facilitate completion of the reviews by July 31, 2002, and would provide an estimate of the amount of time needed to complete the reviews once more details about the systems were developed. However, the vendor completed only one review by July 31 and provided the estimated work hours for only one of the six reviews.

Our outsourcing report also contains a recommendation that Legal be required to review all contracts involving outsourced services in order to help mitigate Legal risk. This contract illustrates how the current process is not ensuring timely Legal review. According to the Acquisition Policy, Legal should review any cost-reimbursement contract with an estimated value of more than $50,000 or any contract over $250,000. We believe this contract should have been reviewed at the outset
because it was, in our opinion, a cost-reimbursement contract with an initial value of more than $50,000. However, the policy does not define a cost-reimbursement contract, and the Procurement Section treated this acquisition as a fixed-price, indefinite quantity contract which does not have a $50,000 threshold for Legal review. In addition, Legal might have been required to review this contract at the outset based on the $250,000 threshold if the initial budgeted amount had included a more realistic estimate of the total cost for all anticipated security control reviews. Procurement Section staff did not ask LEGAL to review the contract until August 2002, when the director of IT requested an additional $277,000, thus raising the total contract value over the $250,000 threshold. As part of its review, Legal questioned the justification for the sole-source award and attempted to verify that the contract's pricing was consistent with the GSA schedule. However, these questions were raised too late to affect any procurement-related decisions.

In addition to the two matters discussed above, other contract administration issues surfaced during our evaluation of this contract which go beyond the scope of the earlier outsourcing report's recommendations. Specifically, we found that the vendor performed work, and that the Board paid invoices, prior to completing contract modifications and that the GSA schedule was not effectively used to obtain adequate competition. To address these additional concerns, we are providing the two recommendations discussed below.

1. We recommend that the Staff Director for Management strengthen the contract administration process by ensuring that (1) contract modifications and purchase orders authorizing additional work are approved and processed before the work is performed, and (2) all contracts clearly establish the responsibilities and authorities of the contracting officer and COTR in accordance with the Board's Acquisition Policy.

As shown in  Figure 1, the vendor performed control reviews, submitted invoices, and was paid before contract modifications and purchase orders were approved to authorize the work and increase the value of the contract. The Board risks incurring unnecessary and unauthorized costs by allowing a contractor to perform work prior to completing the associated contract modifications and purchase orders. Allowing the vendor to perform work prematurely also places the Board in a potentially difficult position in the event disputes arise between the contractor and the Board regarding compensation for the work performed. In addition, we found that when the contract modifications were eventually processed, the documentation was not provided to the COTR.

To avoid incurring unnecessary Legal and contractual risks, the Staff Director should strengthen the controls over processing contract modifications and purchase orders. This could include strengthening language in the Acquisition Policy regarding the proper sequencing of contractual documentation or including language in all contracts that clearly and specifically states that the contractor will not be compensated for work performed prior to formal approval of appropriate contractual documents. The Staff Director should also ensure that Procurement routinely and promptly provides COTRs with copies of approved contractual documents so that the COTRs are aware when additional work has been authorized and funded, thus allowing them to appropriately carry out their contract administration responsibilities.

In our opinion, this contract also established an unusual delineation of responsibilities and authorities between the COTR and the contracting officer. According to the policy, the contracting officer is responsible for assigning and approving all services, and the COTR represents the contracting officer in the administration of technical details within the scope of the contract and in the inspection and acceptance of services provided. Under the policy, the COTR is not otherwise authorized to make any representations or commitments of any kind on the Board's behalf. This contract, however, included language that was inconsistent with the policy and gave the COTR the authority to assign and approve all services. That provision, combined with the absence of terms in the contract identifying which systems were to be reviewed, allowed the COTR to act essentially as contracting officer. Based on our discussions with the contracting officer and the COTR, we believe that it was difficult for the contracting officer-a manager-to challenge the actions of the COTR-a senior division officer. While we believe that this disparity between the procurement manager and the COTR was the exception and not the rule in Board contracts, the Staff Director should take steps to reinforce the procurement manager's authority by establishing sufficient controls to prevent similar occurrences. Such steps could include ensuring that the language of all contracts clearly reflects the division of responsibilities and authorities that is provided for in the policy, or requiring the MGT director to serve as the contracting officer on all acquisitions where another officer will be functioning as the COTR.

2. We recommend that the Staff Director for Management modify the Acquisition Policy to 1) require that exceptions to the use of competitive acquisition methods be approved by the MGT director and 2) incorporate guidance regarding how to use the GSA Multiple Award Schedule (MAS) to competitively select vendors for service contracts.

The policy states that using competitive acquisition methods to the maximum extent practicable helps the Board acquire the highest quality services within the time required and at the best possible value. The policy allows for the use of noncompetitive acquisition procedures, such as a sole-source procurement, when justified in writing and when formal bid and negotiated procurement procedures cannot satisfy the acquisition requirements. A procurement can also be sole-sourced when services are available from only one vendor and no other type of service will satisfy the Board's requirements. According to the policy, the Procurement manager is to approve all sole-source justifications.

In 2001, to conduct GISRA reviews of two systems, the Board followed GSA's ordering approach by obtaining bids from three vendors listed on the MAS. The director of IT told us that two of the three vendors had staffs that were appropriately qualified to conduct the reviews. When one of the two vendors withdrew from consideration, the remaining vendor was selected even though its hourly labor rates were two to four times more than the bidder that withdrew. In 2002, the Board did not solicit bids from multiple vendors, but instead awarded a sole-source contract to the same vendor that performed the two reviews in 2001. The contract was implemented as a sole-source procurement based on the director of IT's justification that 1) the Board had issued a request for proposals for the same service in the summer of 2001, 2) the requested vendor was the only qualified company "interested" in performing the work, 3) the vendor satisfactorily completed two reviews in 2001, and 4) IT now needed six to eight more reviews performed. The Procurement manager approved the sole-source justification.

We believe that the Board should have used the MAS to conduct a competitive procurement in 2002 similar to the process followed in 2001. Based on the short timeframes needed to successfully obtain and process bids, the Board could have expeditiously obtained two additional competitive quotes in 2002 from firms on the MAS. This would have provided additional assurance that the Board was getting best value from the original vendor by providing a basis to evaluate total price. This process might also have identified additional vendors with the qualifications necessary to perform security control reviews, further expanding the potential vendor pool from which the Board could select.

Our discussions with the Procurement manager showed that the director's justification of the 2002 contract as a sole-source award was not challenged. The Procurement manager ultimately approved the contract because he believed that there would be no additional benefits to competing the award since the vendor had previously conducted the work for the Board and the vendor was listed on the MAS. Subsequently, however, Legal staff and the prior MGT director questioned the appropriateness of the sole-source approach because of doubts that the selected vendor was the only qualified firm. The Acquisition Policy emphasizes that competitive acquisition methods should be used to the maximum extent practicable. In our judgment, exceptions to this policy should receive an appropriate level of senior management review and oversight. Therefore, we believe that the policy should be revised to require that sole-source or other exceptions to competitive acquisition methods be approved by the MGT director, rather than the Procurement manager.

Our discussions with Board management and staff regarding this contract also showed that there was confusion on the use of the MAS and whether its use, in and of itself, constituted competition. Use of the MAS is one way for an agency to acquire goods and services based on prenegotiated pricing. However, selecting a vendor that GSA lists on its schedule does not necessarily constitute a competitive acquisition. Additional steps are necessary to ensure that the best price is obtained. GSA's ordering instructions for acquiring services using the MAS state that agencies should obtain quotes from three contractors that appear to offer the best value (considering scope of services offered, pricing, and other factors, as appropriate) and then select the one that meets their needs. This procedure recognizes that while GSA has obtained prices (hourly rates) for scheduled services that it has determined are fair and reasonable, ordering agencies must determine that the total price is reasonable for the specific tasks required by the agency.

To help ensure that future GSA schedule acquisitions result in competitive procurements, the Staff Director should incorporate into the Acquisition Policy basic guidance on using the MAS. The guidance should include information about when and how to use the schedule and how many bids are required. The Staff Director should also consider using the Board's intranet to provide occasional tips and reminders regarding use of the MAS. In addition, the Staff Director should ensure that the new guidance describes what supporting documentation should be maintained. Although the current Acquisition Policy requires Procurement to document the acquisition process in the contract file?including the number of solicitations issued, number of proposals received, technical and cost/price evaluation results, negotiation results, and final contract award summary?the policy does not explain what documentation is required when services are obtained from the MAS. We found that Procurement did not maintain a contract file for the 2001 acquisition, and consequently there was no official record showing that the Board had issued a request for proposal, how the Board had identified the three firms that it contacted on the MAS, or how it was determined that the vendor selected was the only qualified company interested in performing the work in 2001. An official record of the 2001 acquisition would have identified the MAS that was used as a basis for pricing and would have provided answers to questions about pricing raised during Legal's review of the 2002 contract.

ANALYSIS OF COMMENTS

We provided a copy of our report to the MGT director for review and comment. The director's response, issued through the Staff Director for Management, is included in this report and identifies actions that have been or will be taken to address our two recommendations. Regarding our first recommendation, the director intends to work with Legal to clear up ambiguities in contracts and related documents. The director also plans to review current standard contract language to identify any ambiguities that might cause a vendor to fail to understand the risk incurred by performing work without a contract authorization as well as any ambiguities in the description of the functions and limitations of a COTR. In response to our second recommendation, the director indicated that the Board's Acquisition Policy was modified as recommended. Our review of the revised policy found that it now requires the MGT director to approve exceptions to the use of competitive acquisition methods and provides additional guidance on use of the GSA schedule.

The director's response also includes several general comments to add perspective to our report. For example, the response states that our report fails to address the level of importance that senior Board management assigned to performing security reviews in order to meet a statutory deadline and to prevent the Board from exposure to legal, reputational, and political risks. We agree that senior Board management assigned a high level of importance to this effort. However, nothing in the security legislation or implementing guidance required the Board to contract out this function, and other alternatives (such as performing the work in house) existed to complete the reviews and meet the reporting deadline.

The response also points out that our report fails to discuss either the team effort needed to provide best value to the Board or the timing and quality of input from the division requesting the work. We agree that a team approach to procurement is important for ensuring that the Board obtains the best possible value. In our 1996 Report on The Audit of the Board's Procurement And Contract Management Process, we recommended that the Staff Director for Management implement a team approach to the acquisition process by clearly defining and documenting the full acquisition process and the roles and responsibilities that the key members of the procurement team play in each phase of the process. We also recommended in our outsourcing report that MGT provide additional guidance to divisions to help improve the quality of SOWs for outsourced contracts.

Finally, the director's response to our first recommendation states that assigning and approving all services is standard language in Board contracts and that there is no indication that the COTR or the vendor interpreted the language as giving the COTR the authority to assign work not covered by the contract. We disagree that such language is standard. We also continue to believe that the language used on this acquisition, combined with the absence of terms in the contract identifying which systems were to be reviewed, gave the COTR authority that exceeded the COTR's functions and limitations contained in the Acquisition Policy. For example, the COTR assigned, and the vendor performed, additional work for amounts that exceeded the initial value of the contract prior to the contracting officer's authorization and approval. We believe that the actions of the COTR and vendor are an indication that they interpreted the contract language as giving the COTR the authority to assign work not covered by the contract.

Major contributors to this report were Mr. Paul Zacharias, Project Manager; Mr. Kyle Brown, Project Leader; and Mr. David Horn, Auditor. We have provided copies of this report to Board officials and it will be added to our publicly available web site at www.federalreserve.gov/oig. We will also summarize this report in our next semiannual report to Congress. Please contact me if you would like to discuss this audit report or any related issues.

Sincerely,

/signed/

Barry R. Snyder
Inspector General

Enclosure

Board of Governors of the Federal Reserve System

REPORT ON THE AUDIT OF THE BOARD'S AUTOMATED TRAVEL SYSTEM

OFFICE OF INSPECTOR GENERAL

	BOARD OF GOVERNORS OF THE FEDERAL RESERVE SYSTEM WASHINGTON, D. C.  20551
		OFFICE OF INSPECTOR GENERAL
	May 6, 2004

Thank you for the opportunity to comment on the Report on the Effectiveness of Administrative Controls over an Outsourced Contract. The report identifies a need to improve certain controls over procurement actions. As a result of this report, these controls were strengthened in the recent revision of the Board's Acquisition Policy.

The report, however, does not note the importance that senior Board management assigned to the tasks performed under the contract in order to enable the Board to meet a statutory deadline. Failure to comply with this deadline would have exposed the Board to legal, reputational, and political risks; therefore, the procurement had to be expedited.

The report also isolates the work of the Procurement Program, rather than discussing the team effort needed to provide "best value" to the Board when acquiring goods and services. For example, the timing and quality of input from the division requesting the work is not discussed in the report. In May 2002, the division sent Procurement a purchase request with an incomplete statement of work. Because it was necessary for Procurement to obtain a better statement of work, there was a slight delay in processing the requisition. But once an adequate statement of work was received, the purchase order was promptly issued. The report does not acknowledge that the outcome may have been different had the requesting division realized the urgency of the situation and started the process earlier. These are factors over which Procurement had no control.

The report also faults the Procurement staff for not ensuring timely review by the Legal Division. Procurement requested the Legal Division's review on August 22, 2002, when it received the request to increase the contract to an amount that triggered the requirement for a review by the Legal Division. The timeline regarding the Legal Division's review of the contract is not discussed in attachment 1 to the report, Timeline for Acquisition of GISRA Review Services. It is not clear how Procurement could have ensured timely review by the Legal Division, which did not comment on the contract until November 2002.

As your report notes, work was performed before the contract was modified. In government contracting, work performed before receipt of a contract is performed at the vendor's risk.

Division's Comments

Process controls were established before the audit began to help ensure that work is not paid for before a contract is in place. Since that time, payments have not been made unless the relevant contracts are in place. In the case of this contract, payments made before the contract was in place

totaled approximately $285,000. No further payments were made for services, completed in November 2002, until September 2003, after the contract had been properly modified. Had the contract not been modified, no further payments would have been made.

Following are comments to the specific recommendations.

1. We recommend that the Staff Director for Management strengthen the contract administration process by ensuring that (1) contract modifications and purchase orders authorizing additional work are approved and processed before the work is performed, and (2) all contracts clearly establish the responsibilities and authorities of the contracting officer and COTR in accordance with the Board's Acquisition Policy.

The discussion correctly states that "the vendor performed control reviews, submitted invoices, and was paid before contract modifications and purchase orders were approved to authorize the work." As previously mentioned, controls in the information system supporting Procurement have been modified so that payments cannot be made until contracts are in place. Any override for emergency payments must be approved by the director of the Management Division and, in the case of Management Division contracts, by the Staff Director. As a result of improved controls, further invoices for work completed in 2002 were not paid until September 2003, after the contract was properly modified.

Clearly, work was performed before the contract was modified. We will review with the Legal Division the current standard language in our contracts to determine if any ambiguity might cause a contractor to fail to understand the risks incurred by performing work without contractual authorization. The report suggests that the Acquisition Policy be revised so that it "clearly and specifically states that the contractor will not be compensated for work performed prior to formal approval of appropriate contractual documents." In fact we will continue to pay vendors for work performed prior to formal contract approval if a contract is subsequently awarded and if such work would otherwise have been paid for had the contract been in force (such a policy permits emergency work to be performed in high risk situations). The risk that the vendor incurs in this situation is that there may not be a contract or the contract may not cover the full scope of the work, in which case there would be no payment. ³

The report states that, the way this contract was written, the contracting officer's technical representative (COTR) was also given contracting officer responsibilities because she was responsible for "assigning and approving all services." The report suggests that this wording,
which is standard in Board contracts, gave the COTR authority to assign work not covered by the contract; however there is no indication that either the COTR or the company interpreted the

language this way. Attachment 1 to this memorandum is the section, included in all contracts, but not this purchase order, that explains the relationship of the COTR to the vendor. Attachment 1 was written and approved by the Legal Division. Mandatory training provided to all COTRs is designed
to educate the COTRs about their roles and responsibilities. ⁴ We will work with the Legal Division to identify any ambiguity in these roles and revise the COTR document as appropriate.

2. We recommend that the Staff Director for Management modify the Acquisition
Policy to (1) require that exceptions to the use of competitive acquisition methods be approved by the MGT director and (2) incorporate guidance regarding how to use the GSA Multiple Award Schedule (MAS) to competitively select vendors for service contracts.

Both changes have been implemented in the new Acquisition Policy.

We note that similar review work was performed in 2001 to meet congressional mandates of the Governor Information Security Reform Act, and in that case bids were obtained from three vendors, one of whom was ruled out because it did not have the required knowledge, skills, and abilities. A second vendor subsequently withdrew its bid. The reasons supporting the sole-source request in 2002 and the Procurement manager's approval of that request are questioned in the report. The report notes that, "Subsequently, however, Legal staff and the prior MGT director questioned the appropriateness of the sole-source approach because of doubts that the selected vendor was the only qualified firm." It is not clear whether any efforts were made to determine if there were other available qualified bidders. Discussions with the COTR indicate that the vendor selected was significantly less expensive than other reputable audit firms considered in 2001 and that she was not aware of any cheaper alternatives. As an aside, the revised Acquisition Policy would consider this work to be competitive because it was a follow-on award to the award made competitively in 2001.

Attachment

J.5 CONTRACTING OFFICER'S TECHNICAL REPRESENTATIVE
FUNCTIONS AND LIMITATIONS

NAME is designated the cognizant Contracting Officer's Technical Representative (the COTR) who will represent the contracting officer in the administration of technical details within the scope of the contract and in inspection and acceptance. The COTR is not otherwise authorized to make any representations or commitments of any kind on behalf of the contracting officer or the Board. The COTR does not have the authority to alter the contractor's obligations or change the terms and conditions of the contract. If, as a result of technical discussions, it is desirable to alter or change contract terms and conditions, changes will be issued in writing and signed by the contracting officer or his authorized representative.

Some of the types of actions that are within the scope of the COTR's authority are:

a. To assure compliance of the contractor's performance with Section C, the Statement of Work.

b. To perform or cause to be performed those inspections necessary to determine the contractor's compliance with the technical requirements and the Statement of Work.

c. To maintain both oral and written communications with the contractor concerning those aspects of this contract within his or her purview.

d. To monitor the contractor's performance and to advise the Board's contract specialist of any deficiencies.

e. To coordinate the availability of Board-furnished property and services and to provide entry to the work area for the contractor's personnel, as required.

f. To obtain the contractor's proposal for a change order and to relay the information in a memo to the contract specialist.

g. To review invoices and, based on satisfactory performance of the terms and conditions of the contract, to notify the Board's Management Division that a payment should be made pursuant to the contract.

h. To determine final acceptance of services provided under this contract.

Footnotes:

1. See our Report on the Board's Outsourcing Operations (A0207) dated February 2004. Return to text.

2. For one review, the vendor segmented the review into ten separate work segments and products. The total cost for this review was about $191,000, an average of approximately $19,000 per review segment or "small system." The average cost of $71,000 for the other five reviews was calculated by taking the total contract cost ($573,000) and subtracting out general planning ($25,000) and the segmented review ($191,000) and dividing the resulting amount ($357,000) by five. Return to text.

3. Paragraph L.4, Conditions, routinely included in Board contracts (but not in this purchase order), states, "This Solicitation does not commit the Board to pay any cost incurred in the preparation or submission of any proposal. The Board is not liable for any costs incurred in anticipation of a contract award. The Contracting Officer, at his sole discretion, may allow those costs that if incurred after contract award would be allowable." Return to text.

4. Such training is required by the FAR and is provided annually by the Board. Return to text