Review by PLC Public Sector in July 2011
A note setting out the issues to consider when carrying out an evaluation of tenders for a public contract to ensure that the evaluation process complies with the public procurement regime.
While specification of a service, goods or works is the most important part of a public contract once it has been awarded, evaluation is the most important part of the procurement process. It is also the most common cause of complaint.
The manner in which UK public sector bodies are able to procure goods, services or works is determined by EU directives, which have been implemented in the UK via the Public Contracts Regulations 2006 (SI 2006/5) (2006 Regulations), and case law (see Practice note, Public procurement in the UK: Legal framework (www.practicallaw.com/5-383-9734)).
The aim of the EU procurement regime is to create a single market by removing all discrimination against or in favour of bidders. Public bodies cannot simply exercise common sense when choosing what may appear best in particular circumstances. The following key principles of the EU procurement regime apply to all aspects of the procurement process, including selection of bidders and evaluation of tenders:
Transparency: this is not simply about disclosure and openness but also the removal of discretion and subjectivity. Evaluation must be based on objective criteria that are known to bidders in advance.
Fairness: evaluation criteria and the evidence required from bidders must be actually and demonstrably related to the subject matter of the contract and applied proportionately to the stated objectives.
Equal treatment (or non-discrimination): all bidders and potential bidders must be given the same opportunity, based on the same information and criteria, and evaluated in a non-discriminatory manner.
These principles apply to all public contracts, including those which fall below the relevant thresholds in the 2006 Regulations . For further information on the obligations imposed by the public procurement regime, see:
Public procurement in the UK: a quick guide (www.practicallaw.com/9-386-0686).
Practice note, Public procurement in the UK (www.practicallaw.com/5-383-9734).
The main aim of any effective tender evaluation should be to determine the tender which:
Best meets the needs of the public body.
Achieves value for money.
The evaluation should therefore identify the best available and affordable business solution, within applicable legal constraints, through a robust and defensible award process.
Whichever procurement procedure is chosen:
Start planning early: it is best practice to have considered the evaluation criteria at an early stage of the project to ensure that the right requirements are put forward.
Advance disclosure: make the evaluation criteria known to bidders at the first opportunity. Although it is an unpopular idea to share evaluation criteria with bidders, this is a legal requirement (see Advance disclosure of criteria and weightings). In any event, advance disclosure is a practical necessity and a way of showing bidders what the public body really considers important. A distinction should be drawn between:
criteria (the qualities or deliverables the public body wants to secure); and
evidence (the behaviours or characteristics that may support the authority's judgement about the criteria).
Test the evaluation process: identify any potential problems. Once the evaluation criteria are set, they must be adhered to. No adjustment of the criteria or their weightings is permitted unless the procurement is rolled back to the stage at which the evaluation criteria should have been set in the first place and the procurement process re-started (see Use of scoring to evaluate tenders).
Keep an audit trail: document the reasons for using specific evaluation methods or criteria. Authorities must be able to defend the award process (see Audit trail).
The European Court of Justice (www.practicallaw.com/5-107-6553) (ECJ) set out the difference between selection criteria (www.practicallaw.com/1-385-1370) and award criteria (www.practicallaw.com/3-385-1369) in Lianakis v Dimos Alexandroupolis (Case C-532/06):
Selection criteria relate to the tenderer and are used to assess the tenderer's ability to perform the proposed contract (for example, in terms of experience, economic standing and qualifications) (see Selection criteria). Selection criteria may only be applied at the pre-qualification stage in order to short-list bidders to be invited to tender or participate in dialogue, under the restricted and competitive dialogue procedures respectively. Selection criteria may not be assessed during the award stage.
Award criteria relate to the tender and are used to identify the tender that is the most economically advantageous (for example, in terms of price, technical specifications and project management proposals) (see Award criteria).
In simple terms, selection criteria aims to establish "can they do it?" and award criteria "how will they do it?".
For further details of the ECJ's ruling, see Legal update, ECJ ruling on award criteria under public procurement rules (www.practicallaw.com/3-380-4014).
The Office of Government Commerce (www.practicallaw.com/4-383-5270) (OGC) has published a procurement policy note, which confirms that public bodies must make a distinction between selection and award criteria, see Legal update, OGC publishes procurement policy note on selection and award (www.practicallaw.com/6-386-0032).
The recent decision in J Varney & Sons Waste Management Ltd v Hertfordshire County Council [2010] EWHC 1404 (QB) suggests that there is some scope to revert to selection criteria at award stage. In this instance, the judge held that it could be lawful to require information about the financial standing of tenderers (a selection criteria) at the award stage. However, this decision appears to conflict with the ECJ decision in Lianakis, and contracting authorities should be wary of placing too much reliance on it. For more information on Varney, see Legal update, High Court dismisses claim for damages for breach of procurement rules (www.practicallaw.com/1-502-5429). While the Court of Appeal rejected an appeal against the High Court's decision in Varney, it should be noted that this part of the decision was not appealed (see Legal update, Court of Appeal dismisses appeal about Hertfordshire County Council waste recycling contracts procurement (www.practicallaw.com/3-506-5688)).
There are only three selection criteria permitted by the 2006 Regulations:
Exclusion or rejection: bidders must be disqualified on certain grounds, for example, money laundering, bankruptcy and certain criminal offences, and may be disqualified on certain other grounds, for example professional misconduct (regulation 23). However, even where exclusion is expressed to be mandatory, a contracting authority can still permit that bidder to proceed if it is satisfied that overriding requirements in the general interest apply. The pre-qualification questionnaire (www.practicallaw.com/2-385-1379) (PQQ) should list these grounds and require bidders to confirm whether any apply. If they do, bidders should be invited to offer an explanation.
Economic and financial standing: bidders may be eliminated if they do not meet the minimum financial thresholds set out in the PQQ (regulation 24). Contracting authorities usually set the threshold test with reference to the bidder's turnover, that is that the contract value should not exceed a specified percentage of the bidder's turnover. It is important to ensure that the threshold is proportionate in each case, rather than using the same threshold for each procurement, to ensure that bidders are not excluded from participation unnecessarily.
Technical and professional ability: the PQQ may specify minimum technical standards, skill levels and experience but only as relevant to the purpose, nature, quantity or importance of the contract (regulation 25). Regulation 25(2) sets out what information may be required from bidders and taken into consideration for this purpose. This information relates to the bidder and not to any potential solution they may be able to offer. It does not include the implications of choosing the bidder for the project on the following areas:
TUPE (www.practicallaw.com/8-107-7424) (though an assessment of past experience may include consideration of how the bidder has managed TUPE transfers in the past, if relevant);
pensions; or
equal opportunities.
For further guidance on the selection stage, see Practice notes:
The setting of evaluation (that is, award) criteria is not as prescriptive. Subject to the requirements of regulation 30 of the 2006 Regulations, the criteria set out the basis on which the contract will be awarded by measuring value for money (lowest price only or "most economically advantageous tender" (MEAT), that is price plus other factors) converted into scores. Value for money cannot be an award criteria in itself, it is the constituent parts of a bid that will establish whether the bid represents value for money.
When carrying out a procurement, a public body can evaluate tenders by either:
Lowest price only (see below). This is not permitted, however, if the contract is being awarded using the competitive dialogue procedure (www.practicallaw.com/0-385-1375).
The MEAT approach (see below).
The evaluation approach must be decided at the outset.
The lowest price approach is generally only suitable for simple procurements for short-term, low-level services or goods of a standard specification, such as some stationery or linen.
This approach may also be useful in the procurement of a clearly-specified product, service or works, which has sufficient mandatory aspects that would allow a simple choice on the basis of lowest costs. However, it would generally be more appropriate to use the MEAT criteria, subject to an assessment of the weightings to be applied.
Once an authority has determined that it is appropriate to use MEAT to evaluate its procurement, it needs to decide the weighting split between quality and costs. There are no fixed or generally recommended splits. This is a matter for each public body to decide having considered the requirements of each individual procurement. For example:
A very restricted budget combined with a clear and detailed statement of requirements may be more suitable to a higher price weighting. However, it is meaningless to simply allocate, for example 60% to quality and 40% to price. The authority should consider running some dummy scores through its model to ensure that the 60/40 (or other ratio) will deliver the right result, see Use of a scoring model to evaluate the financial element.
Where the level of service to be delivered is key to the public body, such as many consultancy services, it is likely that quality would have the higher weighting.
In some circumstances, it might also be appropriate to have a specific weighting for legal considerations. However, unless the competitive dialogue procedure (www.practicallaw.com/0-385-1375) is used, the public authority can determine the contract terms at the outset and refuse to negotiate them with individual bidders. Where the authority is contracting on its standard terms, it should ensure they are reasonable. Where amendments to the terms are permitted, the contracting authority must determine how these are to be evaluated, and any impact of the suggested change on the other criteria, for example price, must also be considered.
There is no definitive list of quality criteria that should be applied to all procurements, but they must be linked to the subject matter of the contract, that is directly related and proportionate to the contracting authority's requirements. The following may be appropriate:
The criteria chosen should be allotted weightings to reflect what is most important in any particular procurement. Weightings may be exact percentages or a specified range, where this is appropriate in view of the subject matter.
Each broad criteria should ideally be broken down to set out what it is that the public body is looking for to ensure consistent scoring is applied.
If it is not possible, on objective grounds, to provide weightings for the award criteria, it is permissible to indicate the criteria in descending order of importance, but this is unlikely to be the case in most procurements.
Recent UK case law makes it clear that public authorities are expected to disclose, in advance, award criteria, sub-criteria. In Lettings International Ltd v London Borough of Newham [2008] EWHC 1583 (QB) the High Court held that the weighting of sub-criteria should also be disclosed, where known (see Legal update, Court of Appeal allows interim injunction to prevent award of contracts (www.practicallaw.com/4-381-0811)). However in J Varney & Sons Waste Management Ltd v Hertfordshire County Council [2011] EWCA Civ 708, the Court of Appeal held that interpretation adopted in Lettings had been inappropriate and impractical (see Legal update, Court of Appeal dismisses appeal about Hertfordshire County Council waste recycling contracts procurement (www.practicallaw.com/3-506-5688)). In Varney, the Court of Appeal held that If the award criteria and their weighting, and the sub-criteria of the award criteria, have been established and described in the contract to documents, a contracting authority is not obliged to announce in advance the weightings that it intends to apply to the sub-criteria. However any weighting given to sub-criteria must not:
Alter the criteria for the award of the contract set out in the contract documents or the contract notice.
Contain elements which, if they had been known at the time that the tenders were prepared, could have affected that preparation.
Be adopted on the basis of matters which are likely to give rise to discrimination against one of the tenderers.
The OGC's procurement policy note on the selection and award stages of a public procurement reaffirmed the need to disclose award criteria, weightings and scoring to tenderers as early as practicable in the procurement process and no later than the invitation to tender stage. The policy note also states that selection criteria must also be disclosed in advance to interested parties. For further information, see Legal update, OGC publishes procurement policy note on selection and award (www.practicallaw.com/6-386-0032).
The allotted weightings must therefore be stated in either the contract notice (www.practicallaw.com/9-385-1432) or in the contract documents (that is, the tender documents). If the contract is being awarded using the competitive dialogue procedure, the weightings must be stated in the descriptive document (see Standard document, descriptive document (www.practicallaw.com/4-384-2044)).
The EC Treaty principle of transparency may require disclosure of criteria and weightings for procurements that are not covered in whole or part by the 2006 Regulations. On 18 November 2010, the ECJ partially upheld an action brought by the European Commission alleging that Ireland had breached its obligations in relation to the procurement of translation services. The relevant services were "Part B" services that are not subject to the full application of Directive 2004/18. The ECJ concluded that Ireland did not breach the principle of equal treatment and the consequent obligation of transparency by setting the weighting attributable to stated award criteria only after the bids were submitted. However, the modification of the award criteria weightings after an initial review of the bids did amount to a breach of the principles of equal treatment and transparency. For more information, see Legal update, ECJ rules that Ireland breached principles of equal treatment and transparency by altering weighting of award criteria (www.practicallaw.com/2-503-9553).
If some aspects of the requirements are fundamental, a contracting authority may want to include a pass or fail hurdle on these sections. For example, a bus service operator must have appropriate licences in place or a particular aspect of the service must be to a specified ISO standard, as certified by a particular authority. Again this should be set out clearly in the tender documents: both that such a hurdle will be used and how it will be used.
Alternatively, a scoring hurdle can be used where only bids which score a specified minimum on clearly identified and specified key aspects would be considered, for example, using the 0 to 5 scoring system, a contracting authority could specify that only those bids scoring a minimum of 3 in every requirement will be considered. This avoids the risk of a winner which scored very highly in one or more areas but very poorly in another (see Awarding a score for quality). However, contracting authorities should be wary of adopting this approach for criteria which are less important to it, for risk that an otherwise strong bid is excluded on minor grounds.
Contracting authorities may reject and not evaluate the remainder of a tender where it has failed to meet its stated minimum thresholds (see Legal update, Northern Irish High Court dismisses procurement appeal by Irish Waste). (www.practicallaw.com/4-525-6282)
Nobody ever expects a tie, but this can happen. Consider building in potential tie-breakers into the evaluation model and disclosing to bidders upfront how such a mechanism may be used if there is a tie. For example, this may be by:
Setting additional questions to be answered.
Re-valuing (up or down) previously specified points.
Asking for an oral presentation.
A tie does not have to be an exact tie but a statistical tie (that is, a tie within acceptable margins of error) making it difficult, if not impossible, to differentiate between two or more bids.
Authorities also need to consider what methods they will use to evaluate the award criteria and set these out in the tender documents so that all bidders have a fair and equal opportunity of presenting the authority with the appropriate information. Possible methods include:
Written tender (for example a response as to how the bidder will meet each of the requirements of the specification).
Method statements (as above, but the questions will generally be on thematic lines and one question may cover a number of requirements).
Presentations or interviews (typically used to demonstrate a product, for example a software solution. Interviews may also be used to assess softer criteria, for example communication skills and working with the authority, or, in a social or health care context, its service users).
References (as these refer to past experience they should generally be taken at PQQ stage, though they may be used at award stage to support the bidder's proposal for the tendered service).
Site visits.
Sample products.
Authorities will need to decide what percentage of the overall scores each of the criteria carry and how these will be built into an evaluation. For example, an interview or presentation could be used to assess a different part of the technical requirements from the written tender or it might be a means of adjusting the written tender scores. In any event, presentations must be scored as part of the quality or technical evaluation, not added to the final overall quality or price marks or the presentation risks, which would distort the evaluation process.
In a MEAT evaluation, the quality and price scores are converted into percentages in accordance with the pre-set weightings to create a combined score that should show who the successful bidder is.
One of the easiest ways to ensure that evaluation of quality is done consistently is to mark all questions out of a set figure (for example, five) and apply the relevant weighting for that particular criteria to obtain a score. A suggested method for marking out of five would be:
Score | Criteria for awarding score |
0 | Completely fails to meet required standard or does not provide a proposal. |
1 | Proposal significantly fails to meet the standards required, contains significant shortcomings and/or is inconsistent with other proposals. |
2 | Proposal falls short of achieving expected standard in a number of identifiable respects. |
3 | Proposal meets the required standard in most material respects, but is lacking or inconsistent in others. |
4 | Proposal meets the required standard in all material respects. |
5 | Proposal meets the required standard in all material respects and exceeds some or all of the major requirements. |
If this model is used, a bidder meeting the specification in full will only score 4 out of 5 (or 80%) for each assessed requirement. By allowing the fifth mark (20%) for added value the public body is leaving some scope for the bidders to improve on the specification and demonstrate to the authority what else they can provide. In such cases, the tender documents should make clear that bidders are expected not only to meet the required standard but to exceed them, if they are to obtain the maximum available score. However, the authority does not want to award the extra 20% for additions that are no use to it. Therefore, it should consider providing some guidance as to the added benefits it has in mind.
Turning prices into numbers is usually more challenging. A number of approaches may be taken when evaluating the financial element of a tender, but in no case should a simple aggregation be used as this almost always distorts the price scores.
A scoring model (no matter how simple) to create some sort of comparable figure across bids should always be used. The two most common methods of evaluating price are to:
Award 100% to the cheapest bid, then compare that lowest figure as a percentage of the other bids. (See the following two worked examples.)
Disclose a threshold price to bidders, which scores zero, and all bids under that threshold are scored comparatively, inverting the model below. For example, if the threshold is £100, a bid of £60 would score 40% and a bid of £70, 30%.
Outsourcing contract Formula: (Lowest price / price you are evaluating) x 100 = price score for that bidder Bids The bids are based on a fixed price per annum. You have three bids:
Scores Following the above formula, the results are:
|
Domiciliary care contract Formula: (Lowest price / price you are evaluating) x 100 = price score for that bidder Bids The bids are based on hourly rates with no fixed or estimated number of hours required and therefore no overall price. However, the same formula can be used by applying it to the hourly rate quoted. You have three bids:
Scores Following the above formula, the results are:
|
Note that the difference between the bids in the domiciliary care contract is very small and therefore the pricing evaluation for the that contract will have very little impact on the outcome. If both these tenders specified a division of 60:40 in respect of quality and price, pricing would be more important in the outsourcing contract because pricing is not so uniform in this type of service. These examples illustrate the importance of working through some examples based on the sorts of bids which are likely to be received before deciding on the appropriate quality/price weighting.
Note that in addition to scoring a bid price, contracting authorities can also evaluate other financial implications of the bid to come to an overall score, provided this is disclosed to bidders. For example, the costs to the authority if any of its internal resources are required, or, where several contracts are being tendered together, any savings where one bidder wins more than one lot.
Contracting authorities should disclose their intended scoring methodology to bidders in advance.
Often, when evaluating a bid, an answer may be unclear or an evaluator may feel that they need further information to enable them to evaluate the response properly. In this situation, the public body must ensure that it is fair and treat all bidders equally in the way that it seeks to clarify these matters. For example, if two bidders have both been vague in different ways about the provision of a service, both should be asked to clarify their answers.
As a general rule, it is in the public body's best interests to seek clarifications wherever needed, as this helps with selecting the best bid.
Clarifications and responses on clarifications must always be in writing and made through one channel, see Legal update, Northern Ireland High Court dismisses appeal against exclusion of bidder (www.practicallaw.com/6-500-1968).
When preparing to issue a tender, public bodies need to establish who will be involved in the evaluation of responses and ensure that there is sufficient capacity available to meet the required timeframes.
For complex procurements, it is likely that a range of individuals from different disciplines will need to be involved to evaluate the bids properly and it is likely to be a time-consuming task. Where using a team to evaluate bids, each member of the team should:
Be fully briefed about the evaluation methodology that is being used.
Ideally consider the tenders separately before coming together to moderate their scores.
This approach ensures that the risk of potential bias is reduced as far as possible. As with all public sector decision-making, due process must be followed.
If, during a complex procurement, one of the evaluation team needs to be replaced, a thorough and detailed briefing must be given to the new team member to ensure that they fully understand the process that they are stepping into and the evaluation method being used. All of these must be clearly documented.
The public body should treat all bidder responses in a confidential manner during and after the procurement process. This is particularly important where an incumbent supplier may be bidding for a new contract and has access to the contracting authority's premises or systems.
In brief, the following points should be complied with:
All reasonable precautions must be taken to ensure that documentation from bidders and the evaluation materials are kept secure at all times.
Only relevant personnel should be granted access to the materials.
Information received from bidders must not be passed to any third party other than the agreed external advisors.
Copies of the information taken outside of the workplace must be subject to additional care and no copies should be made on any personal IT apparatus.
These precautions are subject to any disclosure obligations the public body has under the Freedom of Information Act 2000 (FOIA) (see Practice note, Freedom of information (www.practicallaw.com/0-200-9452)). However, prior to disclosing any information obtained during the course of a procurement, the exemptions in sections 41 (confidential information) and 43 (commercial interests) of FOIA should be reviewed to see if they apply. The OGC has also provided guidance on what information relating to a procurement is likely to be exempt from disclosure under FOIA, see OGC, FOI and Civil Procurement guidance.
A variant bid (www.practicallaw.com/7-385-1367) is where bidders have proposed an alternative solution to that set out by the public body. The contracting authority must disclose whether or not it will accept variant bids and whether this is mandatory or voluntary. A variant bid is usually only considered when the bidder has also submitted a standard bid.
Variant bids are useful because they allow bidders to suggest innovative ways of delivering the public body's requirements, which may be beneficial. However, the authority should set out clear parameters within which such variants may be made, that is, which areas may be varied. All variant bids should be evaluated using the same criteria as the standard bids and compared on a like-for-like basis. This can often be hard to achieve unless, of course, the parameters set are sufficiently clear.
The public body does not have to accept a variant bid if it is the highest scoring response, so long as it has reserved this right in advance. It is therefore essential to also set out in the tender documents what discretion the public body may exercise in relation to variant bids.
Evaluation is an area that is seeing an increasing number of challenges in the courts from bidders who are unhappy with the way in which public bodies have dealt with their bids. Bidders can potentially claim not only their wasted tender costs but also potential loss of profit for the contract. As recent cases have indicated, it is not even necessary for unsuccessful bidders to prove that they would have been successful but for the evaluation errors. All they have to show is that they have been denied a fair attempt at the tender.
As a result of this increased risk to the public body, it is vital that a clear audit trail of the entire decision-making process throughout the evaluation process is maintained together with copies of all tenders submitted.