The CJS IT approach to Portfolio Management won the 2007 UK Civil Service Financial Management award, is the subject of a case study by Gartner, and has been commended by Booz Allen Hamilton and is cited as best practice in reports to the OECD and European Commission.  This article outlines the approach.  

Introduction

Organisations invest in IT projects and programmes to deliver, or more usually to enable, benefits in terms of regulatory compliance, cost savings, increased revenue and/or performance improvements.  This applies as much in the public as it does in the private sector - for example, the Office for Government Commerce[1] state, "The fundamental reason for beginning a programme is to realise the benefits through change" and the Cabinet Office[2] agree,   "It is only possible to be sure that change has worked if we can measure the delivery of the benefits it is supposed to bring." 

Yet the track record in government is not good - the Public Accounts Committee for example, concluded in 1999 that[3]:

"...for more than two decades, implementing IT systems successfully has proved difficult...has resulted in delay, confusion and inconvenience to the citizen and, in many cases, poor value for money to the taxpayer."

High profile failures in recent years indicate that the situation has not improved significantly and the issue is not restricted to the public sector or to the UK - KPMG for example, report in their 2005 Global IT Project Management Survey that, "project success appears to equate to achieving an acceptable level of failure or minimizing lost benefits."  The issue was summed up most notably by the Nobel prize winning economist, Robert Solow[4], who commented, "You can see computers everywhere but in the productivity statistics."

This is however hardly surprising when as Gartner[5] report, "Business cases are... generally viewed only as documents for gaining funding.  Once approved they are put away....few track the business benefits the projects actually achieve."

By 2004, the Criminal Justice System IT programme (a £2 billion investment between 2003 and 2008) faced a familiar situation - costs were escalating, delivery was delayed and benefits were falling significantly.  Criminal Justice IT's response[6] was to develop and implement an approach that integrates Portfolio Management techniques derived from modern finance theory, with more effective approaches to benefits realisation that embed responsibility within the business rather than with the project.  This approach, which combines robust scrutiny of investments throughout the project life cycle, with one that captures all forms of value created, has borrowed extensively from best practice both here in the UK and abroad - see Figure 1.

Figure 1 An integrated, active, repeatable process

This article outlines the key principles, tools and processes that are, it is argued, applicable to major investments beyond the criminal justice and the IT contexts.

Investment Appraisal

In allocating funding and assessing performance, it is crucial that projects are appraised on a level playing field against agreed criteria.  A first step is therefore to agree a set of ‘Investment Principles' against which all existing and potential projects will be appraised.  The nine CJS IT Investment Principles agreed at Ministerial level are as follows:

• 1. All programmes and projects that are funded by the CJS IT ring-fence budget are governed by the Spending Review Settlement Conditions and the CJS IT governance arrangements.
• 2. We should not fund new projects that do not contribute to the CJS Vision or add value to multiple Criminal Justice Organisations (CJOs).
• 3. What has been started should be finished i.e. funding should continue for those projects that are on time and to budget, otherwise previous investment would be wasted.
• 4. New projects should not be funded unless they can demonstrate a positive return on investment and the benefits are agreed, in principle, by all parties.
• 5. Where possible, options selected should represent: the shortest implementation timeframe; the lowest cost and; deliver a reasonable amount of benefit to the CJS.
• 6. Overspends should be borne by the relevant organisation rather than the CJS IT ring-fence.
• 7. Continued funding will be contingent upon project performance, in terms of meeting delivery milestones and realising benefits, and will be reviewed periodically.
• 8. Projects will not be funded if critical issues are unresolved.
• 9. Projects should not be started if the assessments of their achievability and attractiveness do not meet minimum standards.

The principles of ‘Attractiveness' and Achievability' underpin the approach and reflect an application of the concepts of risk and return derived from Portfolio Theory, to the Project and Programme Management environment where:

• Ø ‘Attractiveness' is appraised in terms of the contribution the project or programme will make to strategic priorities and financial/economic return measured by it's net present value (NPV); and
• Ø ‘Achievability' is assessed as the: degree of innovation and complexity; whether there is sufficient capability and capacity to deliver the project; the adequacy of risk assessment and contingency; and stakeholder commitment.

Investment appraisals are however dependent on reliable data - as Cooper and Edgett[7] say, "The best project selection system in the world is worthless unless the data is sound."  Unfortunately, experience indicates that, as one practitioner says[8], "Business cases contain untested assumptions masquerading as facts".  This view is supported by eempirical research which indicates that optimism bias is a reality i.e. "There is a demonstrated, systemic, tendency for project appraisers to be overly optimistic.  This is a worldwide phenomenon that affects both the private and public sectors...appraisers tend to overstate benefits, and underestimate timings and costs"[9]. 

It is therefore essential that such claims are subject to robust scrutiny.  An independent Portfolio Unit has therefore been established within CJIT to review business cases and evaluate them against the agreed Investment Principles.  This unit triangulates and validates cost and benefits data by:

• 1. Firstly, appraising the cost/benefit case against: the HM Treasury ‘Green Book' rules for economic appraisal (ensuring that costs and benefits are valued wherever feasible and are adjusted for Optimism Bias); the hurdle rates of return specified by the current Spending Review Settlement conditions; and the CJS IT Benefits Eligibility Framework (see below);
• 2. Secondly, ensuring that benefits are agreed, at least in principle, by the recipient agencies, strategic and efficiency planners - for example, the project's impact on each strategic target is rated as: Mission Critical, Highly Desirable, Desirable or Minimal - and these ratings are agreed with the relevant Strategic Planners; and
• 3. Thirdly, using the Proving Model^TM, the only OGC/APMG Gold accredited investment appraisal tool. The model is used to assess projects' attractiveness and achievability using a series of questions derived from a two year research study undertaken at Cranfield University into the causes of programme failure. This research concluded that in the overwhelming majority of cases, these factors were manifest prior to business case authorisation. In other words, failing projects tend not to have brilliant business cases and strong stakeholder commitment - and early identification of these factors has a massive payback in terms of detecting the causes of failure prior to major investment. The appraisal includes a detailed review of project documentation and stakeholder interviews. The output is a summary report - See Figure 2.

Figure 2: The Proving Model assessment

The Proving Model scores are then combined with the financial/economic analysis and benefits reviews in an Investment Appraisal report (see Figure 3).  This ensures that rather than having to wade through 100 page business cases, governance bodies receive a summary focusing on the salient facts - how much is the project going to cost, what benefits are claimed, what is the contribution to strategic priorities and what degree of confidence do we have in these costs and benefits?

Figure 3: The Investment Appraisal Report

Portfolio Management and Prioritisation

Portfolio Management is defined by the OGC as[10], "a corporate, strategic level process for co-ordinating successful delivery across an organisation's entire set of programmes and projects."  The objectives of the process include:

• 1. To obtain the highest return (in financial and/or performance terms) from available resources given an acceptable level of risk
• 2. To ensure balance - in terms of investment types and organisational strategies
• 3. To ensure funding allocations reflect business priorities
• 4. To reallocate funds when performance deteriorates and/or priorities change
• 5. To manage dependencies, constraints and minimise double counting of benefits
• 6. To manage Portfolio-level risk and uncertainty
• 7. To provide transparent reporting on performance

A key element in the process is the management of constraints - and a key factor in this regard is the third ‘A' of Affordability.  Portfolio Prioritisation exercises therefore need to be undertaken taking account of not only projects' NPVs and ‘Attractiveness' and ‘Achievability' ratings, but also ‘Affordability' i.e. projects' relative rankings need to be considered in the context of available funding.  

The assessments of new and existing projects are plotted on an Attractiveness & Achievability matrix to demonstrate their relative rankings (see Figure 4). 

Figure 4: Portfolio Analysis - Attractiveness and Achievability

Consideration however also needs to be given to additional factors such as:

1. The existence of any mandated or regulatory requirements. Such investments are by definition non-discretionary and therefore need to be funded prior to allocations to discretionary projects.

2. Balance across the Portfolio in terms of investment in: the various parts of the Criminal Justice System; supporting the various strategic targets; and in the types of investment - infrastructure and applications.

3. The existence of interdependencies between projects. Infrastructure projects in particular, tend to score poorly in terms of financial return on investment but can enable applications that have strong strategic alignment and positive NPVs (see Figure 5).

Figure 5: Portfolio Dependency Analysis

The output of this exercise is an allocation of funds to existing and proposed investments according to the following categorisation: Invest/Commit; Hold (i.e. invest when funding becomes available); or Reject/Cull.  The conclusions of the prioritisation exercise are then reported in a Delivery Plan which is subject to approval by Ministers.

This is however not just a once and for all activity - formal Portfolio Prioritisation exercises are undertaken at the start of each new Spending Review period (usually every two years) but the Delivery Plan and funding allocations are reviewed at least every six months.  In this way, the CJS IT Portfolio operates a series of "Gates with Teeth" - review stages that operate at two levels:

• Ø At the individual project level (where formal reviews are undertaken at Outline and Full Business Case stage and then throughout the project life cycle) so that funding allocations are: incremental; only guaranteed only to the next review; increased as confidence in outcome grows; and linked to performance.
• Ø At the Portfolio level - prioritisation is re-visited on a periodic basis to:
• § Assess performance in terms of delivery against agreed milestones, costs and whether the anticipated benefits are being realised;
• § Identify and resolve any emerging issues that might prevent the Portfolio from meeting it's objectives; and
• § Ensure the Portfolio remains aligned to strategic priorities.

Active Benefits Management

The OGC estimate that 30-40% of systems to support business change deliver no benefits whatsoever[11].   Experience indicates that the reality may be worse: benefits are often poorly defined in business cases; success in many projects is defined in terms of deployment rather than usage; planning for benefits realisation is rudimentary; and little effective action is taken when benefits are not delivered.  Too often benefits management is seen as an unnecessary overhead that gets in the way of delivery. Yet this attitude is itself at the heart of why so many projects fail to deliver the promised benefits.

The CJIT Portfolio Unit has addressed these issues by establishing a Value Management Office (VMO) with an approach that combines robust scrutiny and validation of claimed benefits with research and analysis to ensure that all relevant benefits are captured.  The approach:

• Ø applies a consistent benefits framework across the programme;
• Ø focuses on benefits throughout the project lifecycle, from outline business case through to post-deployment;
• Ø adopts a joined up approach to the identification and valuation of cross system benefits;
• Ø embeds responsibility for benefits realisation, not only with the relevant projects, but also those in the business able to influence their realisation; and
• Ø includes regular review points to ensure that, if benefits can no longer be achieved in a cost-effective manner, then appropriate action is taken and, if necessary, resources are re-directed to more deserving projects.

Independent Scrutiny and Validation - ensuring Benefits are robust and realisable

Costings are generally subject to detailed rules about what can and can't be counted but the same is not true of benefits.  CJIT has consequently developed a ‘Benefits Management Framework' that encompasses:

• 1. A ‘Benefits Classification Framework';
• 2. The ‘Benefits Eligibility Framework';
• 3. A quarterly Benefits Integrity Check; and
• 4. A framework with clear accountability and responsibility for the realisation of benefits.

Each of the above will be addressed in turn. 

A ‘Benefits Classification Framework' has been developed that values benefits on two dimensions: firstly, by type - benefits are classified into efficiency (cashable and non-cashable) and effectiveness (the economic value of improved performance) categories; and secondly, by recipient i.e. the organisation, group or sector that actually receives the benefit.  In this way benefits are represented in a ‘Benefits Grid' where the horizontal figures are required to reconcile to the vertical totals - see Figure 6.

Figure 6: The Benefits Grid

The ‘Benefits Eligibility Framework' represents the set of detailed rules about what benefits can, and can't, be claimed and how they should be valued.  As such it ensures a methodologically sound approach to measuring and valuing benefits; minimises double counting; allows a consistent approach across the Portfolio - a level playing field; and so enables comparisons over time and between projects.

Benefits claims and reports are subject to a regular ‘Benefits Integrity Check'.  On a quarterly basis, updated benefits reports are prepared by each contributor project - ‘contributor' in the sense that projects contribute benefits to stakeholders (organisations, groups or individuals).  One page formats have been developed to focus on the key benefit information - see Figure 7.

Figure 7: The Quarterly Contributor Project 1 Page Benefits Report

These reports are checked against the ‘Benefits Eligibility Framework' and are reconciled with benefits reports from each recipient organisation - see Figure 8.

Figure 8: The Quarterly Recipient Organisation 1 Page Benefits Report

It can be seen that whilst the focus of the Project report is on the key benefits forecast over the full life cycle, the recipient organisation report is focused on what benefits have been and will be realised in the current Spending Review (3 year) period.   Where there is a difference in the reported benefits, responsibility for approving them ultimately lies with the recipient agency rather than the ‘contributor' project.   This is a fundamental principle that ensures benefits claims are robust and realisable.

Efficiency benefits are also tracked through and agreed with the departmental efficiency planners.

Number 1 in the National Audit Office/Office of Government Commerce's list of Common Causes of Project Failure is: "Lack of a clear link between the project and the organisation's key strategic priorities"  Performance/effectiveness benefits recorded on the Benefits reports are therefore re-cut to show the contribution the Portfolio is making to the achievement of each strategic target - and these claims are validated with the relevant Strategic Planners via Strategic Target Benefits Realisation Plans - see Figure 9

Figure 9: The Strategic Target Benefits Realisation Plan

The output of this quarterly Benefits Integrity Check is a detailed report consisting of the project, organisation and strategic target reports referred to above.  The position is then summarised in a Portfolio level Benefits Scorecard to provide a one page overview of the current position on benefits forecast and realisation - see Figure 10.

Figure 10: The Benefits Scorecard

This process is supported by a governance and accountability framework within which the OCJR Operational Board (CEO level) have accepted accountability for the realisation of benefits in their organisations from the investment in CJS IT.  They are supported by nominated Benefits Realisation Leads (BRLs) who are responsible for validating/signing off benefits claims and ensuring that the benefits they have agreed are actually realised - benefits not just from projects sponsored by their organisation, but from the wider CJS IT portfolio.  These BRLs meet on a monthly basis as the Benefits Working Group to review progress and address emerging issues.

Ensuring all benefits are captured

International research indicates that e-Government initiatives often struggle to demonstrate a positive financial or economic return on investment.  It is not that these investments do not have a return, rather that traditional financial appraisal techniques do not capture all forms of value added.  This puts at risk continued investment and the realization of benefits from the investment made to date.  It is therefore crucial that this missing value is recognised - whether it relates to cross system performance improvements; social and political value; or what is termed foundation value.

Cross-system Performance/Effectiveness Benefits

Realisation of benefits in a cross-organisational portfolio is problematic in that benefits to one organisation may well be dependent on business change elsewhere in the system. The Cabinet Office e-Government Unit have commented that[12], "Business cases were found to be particularly strong on the assessment of costs and benefits related to the lead department itself...However, the identification and quantification of external benefits, for example to users or other departments, were less well handled resulting in business cases that often understated the benefits, and provided an incomplete base for tracking future 3rd party benefits through to realisation." 

This issue has been addressed by the development of a ‘Root Cause Model' (RCM) as a basis for agreeing, quantifying and valuing cross-system benefits.  The model, which is developed from a series of practitioner workshops and is agreed by the Benefits Working Group, identifies the forecast impact of the Portfolio as a whole on the root causes of key problems in the Criminal Justice System and the contribution to Public Service Agreement (PSA) targets.  As such it represents a detailed cause and effect chain from IT project through to PSA target.  See extract from the RCM at Figure 11.

Figure 11: Extract from the Root Cause Model

The model is underpinned by cost and measurement data that enables the performance impacts forecast by the model to be quantified in economic terms - what are referred to as the Combined Effectiveness Impact (CEI) benefits.  These benefits are recorded on the Recipient Organisation and Strategic Target Benefits Realisation reports referred to above - in this way, these benefits are subject to the same validation and tracking mechanisms as the benefits delivered from ‘silo' projects.

The model has also been found to have a number of other advantages:

• It helps identify double counting in project business cases;
• The process gains stakeholder buy in and fosters cross-system understanding; and
• It promotes discussion of how projects can impact on performance and the business changes necessary to realise those benefits

‘Social' Value

A study undertaken for the Australian Government[13] concluded that, "Determining the benefit/cost ratio for e-Government is not straightforward, as the outcomes and benefits are not just financial.  A particular problem for agencies is in identifying and measuring social value."  

Methodologies developed in other jurisdictions also recognise this deficiency, for example:

• Ø the US Federal Government's ‘Value Measuring Methodology'[14] is based on 5 value factors:
• o Direct user (customer) value - Benefits directly realised by users or multiple user groups. For example, time saved, more convenient service delivery/access etc.
• o Social (non-direct user/public) value - Benefits not related to direct users (e.g. society as a whole). For example, improved trust in Government, participation, inclusiveness.
• o Government Operational/Foundational value - Order of magnitude improvements realized in current government operations and processes or those that lay the groundwork for future initiatives. For example, enterprise architecture and improved infrastructure.
• o Goverment Financial value - Financial benefits that have a direct impact on organizational (government service provider) and other federal government budgets via increased revenue, reduced costs or costs avoided.
• o Strategic/Political value - Benefits that move an organization, and government as a whole, closer to achieving its strategic goals and mission.
• Ø the Australian ‘Demand and Value Assessment Methodology'[15] takes a similar approach and also identifies 5 sources of value:
• o Agency benefits/value - Financial (quantitative) e.g. operating cost reductions, increased revenue, improved efficiency and productivity savings, improved effectiveness, improved service or cycle times, and increased staff retention
• o Strategic value (qualitative) - how well the initiative is aligned with the most important outcomes (and political objectives) for the organization
• o Consumer/user financial value - time and cost savings, faster payments and revenue generation opportunities to users of a service
• o Social benefits/value (economic and non-economic) - encompassing both reach and impact in areas of improved quality of life; improved decision making; and more integrated delivery so increasing business opportunities
• o Governance value - i.e. contribution to broader whole-of-government objectives including more open and inclusive government (citizen participation), accountability and improved information availability (transparency)
• Ø The methodology emerging from the European Commission's e-Government Economics Project[16] takes a similar approach in identifying value in social and political terms as well as the more traditional agency-based efficiency and effectiveness categories - see Figure 12.

Figure 12: The European Commission's economics of e Government Framework

All these models include a recognition that social value is a key aspect of the value proposition for e Government initiatives.  The Root Cause Model is therefore being used to estimate the potential value of the CJS IT Portfolio on the economic and social cost of crime i.e. if the model indicates a value to the Criminal Justice System from reductions in re-offending or ineffective trials for example, then the value of this reduction to specific groups and to wider society can be assessed by independent economists. 

CJIT sponsors such research by Home Office economists.  The analysis produced is subject to regular updating.  Confidence ratings are applied to identify areas in which further research would be helpful and the findings are presented in a range of forecasts rather than a single point value.

‘Political Value' and the avoidance of ‘Things Gone Wrong'

Some projects won't necessarily have a positive net present value, for example: politically mandated projects including those designed in response to major system failures; legally required projects; and regulatory requirements.  Such projects will proceed with or without a positive NPV and this reality needs to be reflected in our Investment Appraisal criteria and Value Management approaches. The rationale for such investments is often:

• The avoidance of systemic ‘Things gone wrong' i.e. major failures in the organisation's purpose or core mission, or
• The avoidance of the consequences of not complying with the law or regulatory requirements.

We are therefore faced with the immediate problems of how to measure things that don't happen and how to attribute their avoidance to the project?

There is an implicit assumption that the value of compliance with law/regulations and avoidance of a policy failure represents the net cost of the project.  The CJS IT approach takes this on step further by:

• Ø Making this implicit cost explicit, and
• Ø Using ‘Willingness to Pay' analysis - requiring Ministerial agreement (via the Delivery Plan) that the net cost represents the ‘Political Value' that they place on the project. This requires two further pieces of underlying analysis:
• 1. A cause and effect analysis to demonstrate the rationale for linking the project to the business requirement (approved by the Senior Responsible Officer) i.e. to provide a degree of confidence that the project will address the issue at hand effectively. Key Performance Indicators (both in terms of final outcomes and ‘leading' outputs) also need to be identified; and
• 2. An options analysis demonstrating that the net cost represents the most cost-efficient solution to address the issue (also approved by the SRO).

Foundation Value

Infrastructure investments generally struggle to show a positive return on investment unless they also take account of the applications that will run on that infrastructure.  However, where such applications have yet to be fully scoped or funded, their benefits cannot be booked in departmental efficiency plans.  The research underpinning the US Value Measuring Methodology concluded that[17], "Early investments in e-services are burdened with the costs associated with building required infrastructure and skills.  Cost analyses that do not incorporate foundational value can make calculating and demonstrating a short-term or even long-term value difficult or even impossible.  Decisions made based on these calculations will stifle innovation and make progress toward transforming government sluggish at best."

Gartner argue[18] that resolving this issue requires a business case, "based on articulating how an infrastructure investment enables future investments as well as current projects and activities to better realize their potential or actual benefits."

There is therefore a ‘Potential Opportunity Value' or ‘Foundation Value' that needs to be recognised, but it is crucial that such values are robust and are actively managed to exploit the capacity that has been created.   CJIT has therefore developed a ‘Potential Opportunity Value' model to forecast and manage the potential value inherent in existing infrastructure investments[19].  This model incorporates:

• Ø The Project Opportunity pipeline is assessed to identify projects with potential positive returns;
• Ø Cost and benefit estimates are adjusted for optimism bias;
• Ø Projects are subjected to probability analysis using a standard assessment template to determine the likelihood of funding being provided in the planning period;
• Ø The above analyses are combined to provide an expected value of the pipeline of potential projects.

Assessments will be updated on an annual basis and performance will be monitored in terms of the extent to which potential projects in the Opportunity pipeline are converted to active projects and the ratio of forecast benefits to benefits realised. 

Conclusions

Experience from the CJS IT Portfolio over the last two years demonstrates that realising the full value for investment in e Government and Transformation initiatives requires consideration to be given to the ‘10 Principles of Benefits Realisation':

1. Benefits realisation starts with the Business Case - research shows that failing programmes rarely have strong business cases.
2. Benefits must be placed at the centre of the Portfolio Management and Investment Appraisal processes - funding should be linked to benefits forecasts and project sponsors should be able to answer the question "what benefits am I buying?"
3. Optimism bias is a reality - benefits tend to be OVER stated and are often little more than unsubstantiated assumptions masquerading as facts. Such claims must be robustly scrutinised and challenged.
4. Benefits need to be validated - by agreeing them with the recipients and those who will be responsible for realising them.
5. ‘Book' the benefits early - by adjusting budgets, reducing target unit costs, including them in headcount reductions, or by reflecting them in the organisation's and individual's performance targets.
6. Funding allocations should be incremental and continued funding should be directly linked to benefits realization - regular checkpoints should be built in so that if benefits fall away, budgets can be adjusted accordingly.  Implement regular "Gates with teeth"!
7. Plan and manage benefits realisation from a business rather than a project perspective - benefits are usually dependent on business change and may not be realised until after project deployment has been completed and the project team has disbanded.
8. Benefits need to be actively managed - to ensure that forecast benefits are realised (especially important where those benefits are dependent on business change) and to capture benefits that were not anticipated at the Business Case stage.
9. Capture all forms of value added - efficiency (both time and financial savings), effectiveness (improved performance), foundation/potential opportunity value inherent in infrastructure investments and, particularly in the public sector, wider social and political value - and apply these categories from business case preparation, through investment appraisal and portfolio prioritisation, to benefits realization.
10. Utilise Summary documentation - short summary documents and reports (business cases, benefits reports etc) convey the relevant facts far more effectively than long documents.  Size, in this context, is the enemy of understanding.

Biography - Stephen Jenner

Steve heads the CJS IT Portfolio Unit that was established in 2005 to manage the £2 billion investment by the UK government in modernising and joining up the criminal justice system in England and Wales. As such his responsibilities include: making the case for continued investment to HM Treasury; Investment appraisal; portfolio prioritisation; and benefits realization. Since June 2006, he has also filled the position of Home Office Director heading CJIT, following the appointment of his predecessor, John Suffolk, as the first UK Government CIO.   From April 2008 Steve is working on the pan-government portfolio and benefits management agenda on secondment to the Cabinet Office.

Steve is a Fellow of the Chartered Institute of Management Accountants and Institute of Internal Auditors, and has an MBA from Strathclyde Business School. He also holds a Masters in Applied Criminology from Fitzwilliam College, Cambridge University.

He is a member of the Senior Civil Service and has over 16 years experience in the Criminal Justice System, primarily in the Prison Service where he was Head of Internal Audit (1994-1999) and Head of Financial Accounting (1999-2001). In 2001-2002 Steve enjoyed a 12 month secondment to Queensland, Australia where he led on the implementation of cross government initiatives in the Department of Corrective Services.

Steve regularly speaks at conferences on the subjects of Investment Appraisal, Portfolio Prioritisation and Benefits Management - in the last 18 months he has presented to the OCED in Paris; to the European Commission in Brussels and Vienna; and to conferences in Washington, Florida, Vancouver, London, Rome, Milan and Australia.

Further information and a guidebook are available from the author (stephen.jenner@cjit.gsi.gov.uk or stephenjenner@btinternet.com).

[1] Office for Government Commerce (OGC), Managing Successful Programmes (MSP)

[2] UK Cabinet Office, Successful IT: Modernising Government in Action

[3] Committee of Public Accounts Session 1999-2000 Improving the Delivery of Government IT Projects 

[4] Quoted in J Thorp (2003) The Information Paradox

[5] Gartner (2004) Building Brilliant Business Cases

[6] Criminal Justice IT (CJIT) is part of the Office for Criminal Justice Reform (OCJR) which is a cross-departmental team covering the Home Office, the Department for Constitutional Affairs and the Attorney General's Office.  CJIT has a dual role:  to deliver IT solutions that help join up the CJS and to manage the wider CJS IT programme which also encompasses the provision of modern infrastructure and case management systems in the Police, Crown Prosecution Service, Courts, Prisons, Probation and Youth Justice System. 

[7] Cooper R & Edgett S, (2006) Ten ways to make better project portfolio and project selection decisions,  PDMA Visions Magazine  

[8] Source: Bob Mornan, Executive Director, CIO Branch, Government of Canada. Presentation to the OECD, Paris 6.2.2006 ‘Benefits Realisation: Government of Canada Experience'

[9] HM Treasury. The Green Book Appraisal and Evaluation in Central Government

[10] OGC, 2004, Portfolio Management

[11] Source: OGC Successful Delivery Toolkit

[12] Source: ‘UK Approach to Benefits Realisation Country Report in Support of the e Government Expert Meeting on the Cost and Benefit Analysis of e-Government', February 2006.

[13] NOIE, April 2003,  E-Government Benefits Study

[14] CIO Council, 2002, Value Measuring Methodology - How-TO-Guide

[15] See ‘E-Government Benefits Study‘ [April 2003] from the Australian National Office for the Information Economy (NOIE).

[16] See: e Government Economics Project (e GEP): '‘Measurement Framework Final Version' 15^th May 2006; ‘Economic Model Final Version' 31^st May 2006.

[17] Booz Allen Hamilton, Building a Methodology for Measuring the Value of E-Services

[18] Gartner, How to Demonstrate the Elusive Value of IT Infrastructure in Government

[19] The model is currently being piloted on the Exchange - the IT infrastructure that joins up the CJS and enables information sharing between Criminal Justice agencies.