Analysis

Our research confirms that this issue remains close to the top of digital leaders’ priorities lists. Asked to rate the importance of nine potential elements of a digital strategy (see graph C), survey respondents attending our Vision and Planning workshop put in second place ‘Supportive reforms to departments’ business processes in fields such as data management and procurement’. Immediately behind that, on the bronze podium, came ‘Supportive reforms to systems of project approval, funding, management and governance to suit the needs of digital technologies’. The workshop participants – digital leaders from 15 countries, all working on developing digital strategies – scored these agendas an average of 4.4 and 4.25 respectively, where 4 meant ‘very important’ and 5 ‘essential’.

The salience of these issues illustrates one of the key challenges facing digital leaders: many of the processes and frameworks that govern the development of civil service projects and programmes are neither suited to the needs of digital technologies, nor controlled by digital professionals.

The rules governing spending approval, for example, often demand that project managers set out precisely how a new system will operate before funds are released – yet digital programmes are best developed iteratively, with the right solution emerging late in the build process (see Solution 2).

Along the way, digital project managers may need to commission experimental work to test out different ways into the problem – discarding those that prove to be dead ends. Under many civil service appraisal and accountability frameworks, however, this spending is viewed as money wasted; how then to commission for innovation, accepting that not every idea will bear fruit? (See Solution 3.)

Even when a solution has been identified, government procurement rules often drive up costs and timescales while weakening competition. Onerous qualification requirements can push SMEs out of the market, for example, and mandatory tendering processes can cause long delays as civil servants procure simple tools that could be bought off the shelf (see Solution 1).

With some of these challenges, there are no easy answers. Many digital programmes, for example, involve creating cross-departmental services – requiring two or more organisations to work closely together, sharing the costs and responsibilities. This sits awkwardly with accountability frameworks that hold departmental leaders responsible for every penny of their budgets: they should be sharing decision-making on collaborative projects, but the risk is always theirs to carry.

Some countries have explored ways around this problem. In Singapore, for example, the Ministry of Finance sometimes creates interdepartmental budgets, with two permanent secretaries sharing the responsibility for deciding spending. As Tan Ching Yee, the ministry’s Permanent Secretary, told the Global Government Summit in 2020, this shifts the dynamic “from a negotiation between the Ministry of Finance and them, to one where the people who need to deliver the project figure it out between themselves.”

Such initiatives, she added, enable civil servants to “shoehorn new things into what is quite an established and settled system.” But no country has completely cracked it: as Singapore’s Head of the Civil Service Leo Yip said at the same event, when you have “12 agencies coming together around the table, decisions will often be made by identifying the lowest common denominator.” Ultimately, he concluded, “to sustain a very different way of serving the public, you need a different way of organising. And that’s at the back-end, the middle-end and the front-end.”

As Singapore’s experience shows, it’s possible – given the support of civil service national and financial leaders – to make progress on these challenges. Our research revealed, however, that many finance department leaders continue to discount the harm their rules and systems can cause in digital projects.

At the 2022 Government Finance Summit in Estonia, the messages of our first Digital Leaders report received a chilly reception from the assembled government finance leaders. “Governments’ ways to purchase services and products are cumbersome, but it’s not a real issue,” said one, conceding that tight rules may “cause some extra harm” but maintaining that “it’s quite reasonable to think beforehand about what you’re trying to do. That’s not a hindrance; it’s just normal planning”. Another accepted that rigid systems may lead project managers to “lose time; there may be extra costs”, but added that “with some difficulty, you can still bend [the system] to make it possible”.

Until finance leaders recognise how their cross-government rules can hamper and distort digital projects – with processes designed to minimise risk and protect value for money inadvertently undermining delivery and driving up cost – these systems will remain obstacles to successful transformation. In the meantime, digital leaders can only work hard to raise awareness and understanding, build partnerships with relevant organisations and professions, and make the case for reform.

“The business and the system of government isn’t designed for agile teams,” commented Megan Lee Devlin, Chief Executive Officer of the UK’s Central Digital and Data Office. “But we’re working with Treasury to modernise our approach to funding. We’re working with finance to shift the way they think about Agile when it comes to risk. We’re working with the IPA, our projects authority, to make sure that digital is a really core part of the methodology taught to our project leaders. We’re linking in with our policy colleagues, so that user-centred design, cross-functional teams and data-driven decision-making are more common. And we’re working with the commercial function, to ensure that we make it as easy as possible for government to work with suppliers on the right solutions.”

“There’s a lot to be done,” she concluded. “But our aspiration is to move away from pockets of progress to ensuring transformation, and this is what it’s going to take.”

Solutions

1/ Create a digital marketplace, building an SME supplier ecosystem

In 2010, the UK government spent £16bn (US$19bn) on technology, 80% of it with 20 big companies. To commission a new IT system, it would push vendors through lengthy scrutiny, tendering and bidding frameworks, then select a partner to build out the entire project. The process was rather like a particularly onerous and bureaucratic job application – and with good cause, for buyer and seller would be locked into a close and mutually interdependent relationship for years to come. As in the world of employment, not every such appointment had a happy ending.

Then the Government Digital Service (GDS) was established, and began working to revolutionise the civil service’s approach to managing IT projects. Taking full ownership of their own systems, civil servants would instead build or commission the required components, tools and platforms – choosing the right supplier for each element of the programme. Under this model, they’d run many smaller procurements instead of a few huge ones, and seek to buy from SMEs – which were often more innovative, flexible and economical than the IT multinationals.

The traditional procurement model, however, put many barriers in their way. Required to undergo lengthy and expensive pre-qualification and bidding processes, SMEs simply couldn’t afford to get involved; and every procurement was bespoke, demanding a fresh set of spending. Meanwhile, ‘due diligence’ rules placed heavy emphasis on a track record of delivery for civil service organisations: the outcome was a self-fulfilling prophecy in which only government suppliers could supply to government.

On the other side of the fence, many public sector buyers were now trying to buy very similar services: Cloud capacity, for example, or the software to automate basic processes. So they too had much to gain by pooling demand, accessing tools developed for other organisations and aggregating their purchases to drive down unit costs. Under GDS’s digital model, the job application approach to procurement was no longer appropriate. What was required was a marketplace, in which buyers could browse the wares of many suppliers – so GDS built one.

Launched in 2014, the Digital Marketplace enabled suppliers to present their services to any public sector buyer. So vendors only had to go through the pre-approval process once to reach the whole market, strengthening competition; another barrier was removed by reforming the due diligence requirements to focus on capabilities, rather than customer history. Now buyers could purchase a much wider range of tools and services in a thriving, transparent market, aggregating orders where it made sense to do so; meanwhile the Marketplace supported the development of an entrepreneurial, SME-based tech sector, boosting economic growth.

By 2020, the UK government’s technology infrastructure and online services had much improved, while its IT spending had fallen to £9bn (US$10.8bn). And in 2021-22, £3.9bn (US$4.7bn) of this money was spent through the Digital Marketplace, some 40% of it with SMEs: that year, 2080 suppliers signed contracts with 2133 customers – 398 of them in central government.

Introducing the Digital Marketplace – along with supportive process reforms – has transformed the UK government’s IT purchasing, enabling departments to buy the tools and services required for digital transformation. The Marketplace is much more than simply a better way to buy: by cutting waste and time out of procurement processes – for suppliers and customers alike – and stimulating the development of a new supplier market, it has played a pivotal role in supporting the government’s embrace of digital and data technologies.

2/ Adapt funding and project approval systems to support Agile methodologies

In the old world of IT projects, civil servants would plan in detail how a new system would operate, then commission a major supplier to build it. This clarity about the output enabled budget holders to check that the delivery plan met value for money requirements, ostensibly fulfilling their obligation to protect taxpayer funds.

Frequently, however, project managers would find during the build process that they had specified the wrong output – often because they’d based the system’s design on ill-founded assumptions about user behaviour. And because systems’ final operation had been set in stone – typically with costly financial penalties for in-flight changes – project managers proved reluctant to raise the alert if things started to go awry. As a consequence, many IT projects were either abandoned at massive cost, or produced systems that simply did not work in the messy, unpredictable real world of human behaviours. Project sign-off processes designed to protect value for money had created the most perverse of outcomes: by hard-wiring errors into inflexible plans, they contributed to project failures and vast investment write-offs.

Digital professionals’ response was Agile project management, which shifts the focus from final system design to the behaviours of different user groups. Defining ‘personas’ for each type of user – managers and operators as well as service recipients – Agile developers sketch out various user journeys, then test and iterate their way towards a final system. Ensuring that projects are built around real-world behaviours rather than planners’ expectations, Agile is a far more reliable way to build out complex digital systems. Under traditional approaches, the riskiest projects would be asked to produce the most detailed and comprehensive plans; but the more rigid the plan, the more difficult to react to the unexpected. Because Agile permits project managers to react to new information and changing circumstances, it is at its strongest where the risk is greatest.

To support wider use of Agile in government, spending approval and project management systems – typically owned by nations’ finance ministries or functions – should be reformed, replacing the deceptive comfort blanket of detailed forward plans with a more mature and realistic approach to managing risk. They can, for example, be reshaped around outcomes rather than outputs, with tests focused on project managers’ and suppliers’ skills and expertise rather than the comprehensiveness of their delivery plan. Meanwhile the release of funds can be tied to the creation of key functions and user journeys, rather than milestones in the construction of a predefined system. Project managers and budget holders across government also need training in the benefits and risks of Agile methodologies, and support in applying them intelligently and appropriately.

Such reforms, however, must also recognise Agile’s limitations and drawbacks. The lack of clarity at the outset on how systems will eventually function makes cost estimates uncertain – putting budget holders in an uncomfortable position – and creates problems in highly regulated environments, where the final service must comply with legislative requirements. The simultaneous development of multiple user journeys creates the risk of duplication, leading to unnecessary waste. And because civil servants must effectively estimate the required functionalities during projects’ early phases, they’re on the hook for the price of any requirements that emerge later; so the approach can transfer risk from private suppliers to the public sector.

Some of the solutions lie in emerging ‘hybrid Agile’ methodologies. In the phrase’s most common usage, this simply means combining Agile techniques with more traditional ‘Waterfall’ project management practices. Project managers can, for example, use Waterfall for planning, design and requirements definition, but undertake development work by testing and iterating in ‘sprints’.

Other approaches maintain more of pure Agile’s flexibility. Initial scoping work can, for example, extend from defining the users and their goals to forecasting the functionality required across the system. Project owners can then estimate the total workload required, generating a predicted cost: this enables them to write contracts that pass delivery risks to private suppliers, while providing budget holders with more reliable cost estimates.

It is not realistic to expect a rapid and radical transformation of government spending approval and project sign-off processes – yet they must evolve to meet the requirements of digital, permitting and supporting the project management methodologies most suited to today’s technologies. When applied to traditional IT projects, these systems have too often increased the risks they claimed to reduce; in a digital world, they are still less appropriate. Finance, project management and digital professionals will not find it easy to agree a way forward here; but if they fail to do so, finance leaders will see more money wasted, project managers more failed projects, and digital leaders fewer genuinely digital services.

3/ Innovate in commissioning, and commission for innovation

As the Digital Marketplace (see Solution 1) illustrates, reforms to procurement and commissioning can dramatically increase the potential for digital transformation. And in recent years, digital leaders around the world have found new and inventive ways to buy staff, services and support – strengthening their ability to assist departments, adapt to changing circumstances, and commission experimental and innovative work.

Where nations have strong tech sectors but scarce in-house digital skills, for example, they can buy specialists’ time or ‘sprints’ rather than commissioning new products or services. Andri Heiðar Kristinsson, Chief Executive Officer of Digital Iceland, explained that many departments “don’t have the in-house capability” to manage large-scale transformation projects – “so at the moment, we’re stepping in and doing a lot of that for them.”

A unique tender gives Digital Iceland access to a set of private companies that can be deployed to support agencies, supplying highly-qualified teams to deliver individual ‘sprints’ within host departments. “So if I have a new project that looks good, I can deploy a team within a matter of days,” Kristinsson said. “That’s completely changed how we do things.” The Israeli government has long taken this approach a step further: only its most senior digital leaders are civil servants, with most of the worforce comprising contractors – many of whom move between the public and private sectors, building skills and connections on both sides.

Governments can also adjust their use of contracts to increase agility and flexibility. Contracts can, for example, reserve the right to run multiple suppliers simultaneously – introducing competition during the development process – or include prenegotiated terms and conditions permitting their extension to other government departments.

It’s even possible to commission for experimentation – traditionally a difficult task in government, where spending that doesn’t generate a concrete outcome is often viewed as money wasted. In British Columbia, for example, the provincial government’s ‘Sprint With Us’ scheme funds suppliers to work on ways to address a specified problem, enabling commissioners to take the best forward. The UK’s Innovation Fund takes a similar approach: up to five vendors are paid to develop a solution, with the two most promising winning further work. Recognising the need to build redundancy into experimental and explorative projects, such schemes represent a more mature approach to value for money.