Delivering SOA Across Projects: Dealing with Shared Dependencies & Risks

I posted not long ago about professionalising the planning and delivery of SOA initiatives, making realising them more than just a gamble based on theory and good intentions.

One of the biggest pieces of this puzzle often seems to be dealing with the project delivery issues. The industrialisation of SOA if you like. The key issue is that many are not ready to deal with the shared and cross-project dependencies, the additional technology complexities, and the cross-project risk mitigation demands that enterprise-level SOA creates. Such issues create havoc with the conventional project delivery mechanisms and mindsets of the moment, but without project delivery being on-board, an enterprise-level SOA is extremely difficult to scale up to the level required for it to start paying off.

IT Project Delivery
Modern IT delivery (pre-SOA) practices that combine both robust project management and proper diligent delivery management have improved significantly over the incarnations of the 90s and before. They're particularly stronger in areas like governance, visibility and indeed success rates, learning lessons from mistakes of the past. However they've been shaped by a diet of scope definition, issue identification, risk mitigation, and industrialisation of production in traditional systems-centric world. Where each system and project has its own direct business case. Of course its not perfect, for example the delivery models of many are not as mature as they should be in how they effect global sourcing and production in order to take advantage of off-shore production whilst maintaining quality and efficiency. However, ignoring globalisation, many of the basics of today's delivery get turned on their heads by cross-project SOA initiatives.

Real cross-project SOA introduces dependencies between projects who are sharing services, coexisting in environments, or reusing central assets. It also generally means that issue resolution and risk mitigation within a project is no longer within that project's own hands. And in seeking to optimise the technology across projects, it mostly means that each project is sub-optimised, increasing the 'number of moving parts' of technology (and therefore generally the cost) beyond what is needed to realise the project's business case. A situation which often isn't exactly welcomed by the leadership of the projects concerned. And of course there are many more such changes, but you get the idea.

Some organisations have laudable ambitions for the use of architecture in reversing years of IT duplication, proliferation, conflicts and silos that business-unit-centric or project-centric technology deployment has caused, but they embark on it without a proven or complete plan as to how to deal with the road-blocks, or handle the stakeholders who's incentivisation or behaviours lie elsewhere.

1. Project Effort and Cost
For example, you have the additional factors in estimating the effort required in a project and allocating its cost. Even where a project has agreed to take on the additional technologies and development costs of the new middleware (e.g. ESBs) needed to effect SOA it will need more extra budget provision just for the dialogue needed with the central team doing the architecture and technology-usage governance (be they an EA team or called something else), and participate in the joint service designs and testing. Let alone the change control needed if a project has to comply with a late emerging technology standard, or extra regression testing required because of a change request in a parallel project which shares services. And this is on top of the actual effort taken to build the services that they need to build, and effectively feels to the project concerned like time that is not value adding and is not being spent getting on with the solution.

As often is the case, a commercial contract between two parties focuses attention on things that would otherwise be hidden, and if a 3rd party implementer is involved in delivering a particular project, these additional provisions will need to be surfaced up front, and potentially put into the contracting. That's not a nice conversation to have, but infinitely preferable to a much later conversation justifying changes in budgets when it's too late to do anything about it.

2. Design, Configuration and Release Management
There are also the additional complexities of service design, configuration, and release management in cross-project delivery. Firstly, the environments. On one hand there are more technologies that require environments (to develop->test->cut-over), and then all the technologies (new or old) require demarcation between the (develop->test) environments within their own project's control, versus the (testing->cut-over) environments in common shared environments.

Then secondly there is the new design governance process required to collectively agree a shared design that multiple projects work towards, to manage requests for changes to it during development and when its live, to coordinate integration of parallel work-streams, and to sign-off completion of delivery to the pre-agreed design. That last one is particularly troublesome as many 'design committees' that may exist around an SOA initiative may take on the design sign-off responsibility, but are often not resourced or mandated to sign-off delivery to the design. And yet without that ability the projects are at the mercy of factors outside of their control (and timeline) for sign-off, which may be incompatible with the obligations they have to the business, and their business cases.

3. Risk Management
Even with an appropriate configuration management process, and the appropriate bodies with the appropriate mandate to play the roles in it, it still is more than a little problematic to deliver federated cross-project SOA. Particularly with parallel delivery (as opposed to sharing only assets that have already been deployed) which is probably the most ambitious SOA delivery model.

Management of the risk profiles involved causes significant uncertainties for the commercial models each project is using to manage its cost. Fundamentally the budget that any project uses will be based on a certain scope, certain assumptions, and certain known risks for which it will allow contingencies. Whether there is a 3rd party prime contractor involved, 3rd party subcontractors, or only internal heads involved will affect what type commercial model is used, and therefore how the cost and contingency are estimated. But whatever the case, the project will be carrying a certain risk profile that it will look to manage so as to keep its commitment to deliver.

Of course the dependencies of the project on central activities, on existing seemingly vague assets, and on other projects activities, creates additional risks that existing techniques find it difficult to identify, and which are not within their control. This means that projects are potentially exposed in carrying risks that they find it difficult to estimate contingencies for, and that they are unable to mitigate.

Shared Services (...for Delivering Shared Services!)
But these are of course not really new problems. And they're not really peculiar to SOA. They are an aspect of executing on any shared initiative above the level of a project and business unit (BU), which is dependent on the project or BU to realise it, and therefore applies across those projects and BUs. Ultimately this boils down to being an aspect of achieving objectives across reporting lines, in a situation where those objectives include indirect (federated) business cases, in addition to the (projects' own) direct business cases.

The techniques that has most successfully been used to solve similar problems before in previous generation of IT are simple operating model concepts widely used in other areas of business, taught in most b-schools, and written about many times. They're shared services (, although I'll use the old IT term 'centre of excellence' here to save confusion between the shared services in SOA delivery from the the shared services of service oriented architecture itself).

A centre of excellence (CoE) or competency centre for SOA delivery allows the projects to all sub-contract the production of the (SOA) services to that CoE and therefore to exchange the risk that they can't manage themselves for a service-level agreement (SLA) that they can manage to. It also allows the CoE to manage the internal dependencies between the different requests being made of it, formalising and managing th change on contracts with its customers (the projects) so that it can coordinate the cost and SLAs to the risk it carries at any time. It also of course allows the CoE to drive centralised knowledge management, developing its own capabilities and professionalising its own working practices using its own revenue stream, and gaining from the economies of scale on the supply side that no one project could have experienced on its own.

Its great that its becoming acknowledged that an enterprise-level architecture capability (although often not a traditional EA group with a traditional EA framework) is key to being able to drive cross-project technology and architecture strategies and improvements. In such scenarios that EA capability is in some ways acting like a shared service for the design phase (and to a more limited degree, for the preceding demand management phase). But that is only a part of the lifecycle. A shared design is much harder to make it into reality without some level of shared delivery (construction) and shared operations (run and support). Sounds simple when it's said like that.

Technorati Tags: Enterprise IT Project Management IT Delivery Shared Services Center of Excellence Competency Center Enterprise Architecture SOA Services Architecture Service-Oriented Architecture