Virtually every year, the Government Accountability Office’s (GAO) annual report on bid protests includes “faulty technical evaluations” as one of the five most common reasons for successful protests. In simple terms, this means that if a protest is to be supported at GAO, there is a good chance that the government watchdog will find that the agency has not evaluated the protester’s and/or grantee’s technical proposal. in accordance with the evaluation methodology disclosed in the solicitation. It follows, of course, that more complex assessment schemes (i.e. those involving a multiplicity of factors, sub-factors and weighting schemes) carry a proportionally higher level of risk. that the agency’s evaluators are wrong. The recent GAO decision in AT&T Mobility, LLC provides such an example and is a useful case study for entrepreneurs.
The balance between streamlining the evaluation process and properly performing a qualitative analysis as part of a best value purchase (BVTO) is at the heart of this case. Here, the GAO supported AT&T Mobility’s protest and found that the US Secret Service was relying on a dossier devoid of any meaningful qualitative analysis under the technical factor, and instead evaluating bidders based on a success matrix. /failure which had been included in an earlier version of the solicitation, but which was withdrawn by the Agency as the acquisition progressed. The Secret Service evaluation criteria included additional non-price factors (transition and corporate experience), which should have been evaluated on a BTVO basis, but were also evaluated on an acceptable/unacceptable basis. AT&T Mobility also argued that there were pricing errors. Although the GAO acknowledged that the pricing factor errors may have been innocuous given that AT&T Mobility would remain the most expensive bidder, it concluded that due to the erroneous assessment of non-pricing factors, it was possible that AT&T Mobility was harmed by the errors. Ultimately, the GAO found that the Agency disregarded its stated BVTO rating system and instead awarded the contract based on the lowest technically acceptable price (LPTA).
The decision highlights important considerations for contractors to take into account when deciding whether or not to file a protest.
- Consider the scope of the assessment: The way a solicitation is written can increase the likelihood of a successful protest. Complicated procedures are likely to produce uncertain results that may be worth investigating post-attribution. The solicitation in question called for an evaluation of the technical factors of the ability to meet 112 different basic technical requirements. A BVTO assessment assessing 112 the substantive technical requirements are a logistical nightmare for an assessment team, and it’s no surprise the Secret Service got it wrong. While FAR Subpart 8.4—which has been used here—provides agencies with a streamlined process for procuring certain commercial supplies and services, agencies are still required to evaluate in accordance with the criteria set forth in the solicitation. Often an agency’s desire to ensure that contractors are able to meet all of their technical requirements is reflected in the bid solicitation evaluation criteria. However, a solicitation that requires the evaluation of a long list of technical requirements – and a related cross-factor trade-off based on those evaluation results – can make it more difficult for the evaluation team to perform an analysis. adequate quality of the bidders’ proposals. Thus, when reviewing an agency’s award decision (as disclosed in a debriefing for FAR Part 15 procurements or a “Brief Explanation” of contract award for FAR Part 8 procurements), a disappointed entrepreneur should carefully consider the depth and breadth of the evaluation criteria disclosed. against the stated evaluation results and ask a central question: “Does this make sense?” If the answer is “no” or “probably not,” consider filing a protest. Remember that the devil on an agency’s shoulder can convince the review team that the agency can have it all. The agency can sincerely believe that complex and lengthy assessment criteria are necessary to meet existing and emerging requirements and that its assessors will administer flawless assessments. As with most things that seem too good to be true, by the time difficulties become apparent, the agency may find themselves with less than ideal options. The agency may be forced to figure out how to pull off a Hail Mary, in which it attempts to evaluate offerors according to complex and time-consuming evaluation criteria while meeting its deadlines and adequately documenting the case. When an agency is forced to choose between competing priorities such as meeting internal deadlines and performing sufficient qualitative analysis, the temptation to rationalize in a way not contemplated by the solicitation may become too strong for it. can resist.
- Pay attention to the changed requirements: When an agency modifies a solicitation – and in particular when the agency revises its award methodology – contractors should ensure that the agency has evaluated the proposals in accordance with the final version of the solicitation. Here, solicitation originally articulated on a pass/fail matrix would be used. The final version of the RFP removed the pass/fail evaluation matrix and stated that the ability to meet 112 baseline requirements would be considered in the evaluation of technical factors. The source selection decision indicated that the Agency had considered the pass/fail matrix in its decision and in the basis for attribution – a clear red flag that the Agency had not assessed in accordance with the most recent version of the solicitation. Agency assessments conducted under outdated solicitation requirements can be attributed to a variety of causes. Answers to clarifying questions may alter the solicitation, but the evaluation team may not consider the answers when evaluating. Changes may be made by the procurement team but not adequately communicated to the evaluation team. There may be issues with version control of documents or there may be staff turnover which results in loss of institutional knowledge. Contractors should therefore be careful about using outdated language (or explanations that are based on outdated requirements), as this may indicate that the agency has not assessed in accordance with the most recent version of the solicitation.
- Understanding the debriefing documents: When reviewing reporting documents, contractors should be aware of the procurement procedures used and ensure that the information provided meets minimum regulatory requirements. FAR Part 15 debriefings should detail: (1) any material weaknesses or deficiencies assessed; (2) the overall technical score of the successful Offeror and debriefed; (3) the evaluated cost or aggregate price of the successful and debriefed Offeror; (4) overall classification, if applicable; (5) a summary of the rationale for the award; (6) for commercial products, the make and model to be delivered by the recipient; and (7) reasonable answers to relevant questions as to whether the source selection procedures contained in the RFP, applicable regulations and other applicable authorities were followed. certain contracts, and FAR 8.4 requires a brief explanation of the award decision. Regardless of the operational regulations involved, the reporting documents should provide the contractor with the rationale for awarding agency at some basic level. If the debriefing provides only conclusive and/or superficial information, the lack of detail may indicate that the agency did not properly execute and/or document its decision. determination. In making its award decision, the Agency had to take into account the underlying bases of the ratings and the advantages and disadvantages associated with the competing proposals. The Agency’s decision that the competing proposals were technically equivalent should have been based on a documented qualitative assessment of the proposals, but none was found in the evaluation file.
- Corrective action: When a protest is sustained, GAO generally recommends that the agency take corrective action. Here, the GAO gave the Agency a choice between (1) re-evaluating the proposals in accordance with the current solicitation requirements and (2) re-evaluating its requirements and method of source selection. The GAO also recommended that the protester be reimbursed for protest costs, including attorney’s fees. When determining whether to file a protest, consider the range of potential remedies at stake. Protest arguments that, if successful, can be addressed by fixing only minor issues are much less likely to result a change in the award decision, and therefore contractors may wish to give up the battle in the first place. That said, consider moving forward if the alleged deficiencies are of a more fundamental nature and could potentially alter the award decision once corrected.
- Compliance Note: While not a key issue in this case, the GAO decision indicates that a third offeror was eliminated from competition due to compliance issues. Agencies often use minimum mandatory criteria as a way to reduce the number of proposals that must be evaluated, which in turn allows the agency to streamline the evaluation process. Contractors should ensure that their proposal demonstrates compliance with all affirmative compliance requirements and remember the old adage that “an ounce of prevention is better than cure” when developing their proposals. An agency is usually ruthless when an offeror fails to meet basic requirements. If a compliance requirement does not make sense, contractors must ask questions within the time frame specified in the RFP. Any ambiguity must be brought to the attention of the agency before the closing of the tender.
Basically, this case provides another example of the potential usefulness of the protest process and further confirms that contractors should never be afraid to ask an agency to “show their work” when providing their final response – c ie the award decision – in complex procurement. Remember that 1+1 never equals 3.