Managing Uncertainty in Research & Design
Creating the radically new is hard. The biggest strategic decisions that shape the fundemental characteristcs of the final solution are made at the earliest stages of a new project, when you have the least information. This is why the blank sheet of paper is both so exciting and so intimidating.
Any project developing a new commercial product or technology needs to:
Understand & optimise the user and business value propositions of an idea.
Demonstrate that the idea is technically feasible and commercially viable.
Quantify the idea in such a way that it can be optimised, validated and ultimately legally protected.
It is obvious that to move a project forward, decision need to be made; to make good decisions, good information is required. To understand how best to gather that information, consider the scenario where you need to find your keys. You could randomly search different locations in the room, or you could divide the room up into a grid search each square of space in turn. Both methods could clearly be very inefficient. This example might be silly, but it is often how the scientific method vs a designer’s intuition are viewed. Both would eventually yield good information, but both would be very inefficient. A better method might be to first remember where you saw your keys last or locations you have put them before and start looking in locations you have been in since; if this does not work then you could methodically search the area around each of these locations in an ever expanding radius. Applied to R&D projects, this more nuanced approach is one that blends both creative leaps and methodical research.
There are a number of processes for managing environments of extreme uncertainty when developing new products and technologies, all of these processes essentially aim to clearly articulate questions, the answers to which would provide information that enables a decision to be made. Though they differ in the specifics of their format, each is intended to guide you through that basic loop, each time moving the project forward. Beyond this they also:
Ensure quality by providing a template to ensure nothing is forgotten and prompt you to look at the problem from different perspectives.
Enable better decisions through encouraging an objective appraisal of the completeness and quality of the data available.
Provide clear documentation that prevents a project retracing old ground and a resource for gaining legal protection.
Continuously reassess key strategic elements to ensure not just the project is successful, but it delivers a successful solution.
Track resources (people, equipment, time and money) and outputs (project impacts on wider business KPIs), ensuring synchronisation, allowing for better forecasting, and enabling wider strategic decisions in the organisation.
Align senior stakeholders, team members and departments around a stated purpose and clearly define roles and ways of interacting with a project so large teams can easily work together.
Incrementally build certainty by increasing the resolution of the key project deliverables required at each stage and managing any strategic changes that occur once a project has started.
In my experience, these processes can be broadly split into two main types of methodology that help to do this. The first type is an umbrella framework that defines a project and then breaks its development down into stages and the second is a framework for the delivery of specific work packets within that project umbrella.
PRINCE2 is a good real world example of this the former, PMP and PMI’s PMBOK also share many similarities and waterfall or critical path type methodologies could also be grouped here. PRINCE2 is a general framework intended to be adjusted for the specific organisation and project. It emphasises dividing projects into stages which are easily controlled by a set of defined processes and documents in order to deliver a specified project goal within specified constraints while optimising for six performance goals. The six performance goals are scope, time, risk, quality, benefit and cost and are considered in the decision making process at each stage boundary [1]. The real benefit of PRINCE2 in the context of managing uncertainty is that it formalises an agreed definition of what a project hopes to achieve as a document, which links to all other documents that drive the project. Any changes therefore have a clear impact that can be traced and managed.
![An overview of the PRINCE2 project processes by Ariel Sheen [2]](https://images.squarespace-cdn.com/content/v1/5f659760b0847f6b53bf9134/1600553705373-S2VLWC5UJFMCK9WBCTNF/PRINCE2.png)
An overview of the PRINCE2 project processes by Ariel Sheen [2]
Scrum, Kanban and other agile methodologies are examples of the latter methodology type for work packet delivery very commonly used in software. I have found that processes based on NASA’s Technology Readiness Levels offer a better solution for hardware and early stage research projects which have greater uncertainty. The TRL is most useful when there is no existing process and you have a question that is difficult to answer. The most important aspect that it adds over the others mentioned is a peer review appraisal of work done, confirming the question was answered in a valid way that supports the conclusions drawn; this mechanism also provides cross team pollination of knowledge.
TRLs were developed by NASA during the 1970s as simply a means of assessing the maturity level of given technologies. More broadly, they can be used to assess the maturity level of any type of development project. They have since been adopted by SpaceX, Airbus, Rolls Royce, Lockheed Martin and the European Space Agency. In 2010 all EU funded research and innovation projects were advised to adopt the scale by the EU Commission and many consumer companies now use TRLs to manage R&D including Dyson, John Deere, Bombardier, General Electric, Carl Zeiss, Baxter and Fujifilm [3]. Practically, TRLs are a scale from 1 to 9 with 9 being the most mature technology. To say a project is at a certain level of maturity, you must have met a number of requirements. This is policed by a peer review system, when you want to move from one level of maturity to another, you must show your work to your stakeholders and independent peer reviewers and they must all agree you have met your requirements, in a valid way, and have a good plan for moving forward.
T1: You identified a research challenge of defined strategic interest and converted it into deliverables. You have a plan to get to the next TRL.
T2: You figured out who your key stakeholders are and set an agreed project scope. You wrote functional & non-functional requirements for your deliverables. You have a plan to get to the next TRL.
T3: You have identified project risks and set target values against each of your functional requirements. Now you can ensure what you end up delivering will perform as required. You have a plan to get to the next TRL.
T4: You have test methods for all of your functional requirements and have generated and compared potential solutions. You have conducted initial IP landscape searches. You have a plan to get to the next TRL.
T5: You have developed a detailed understanding of the variables that influence performance in a stand-alone sub-system and selected a solution. You have a plan to get to the next TRL.
T6: You tested the solution against all your test methods, optimizing and validating it and documented a final specification. You have checked to see if you generated any IP. You have a plan to get to the next TRL.
T7: You have integrated your optimized sub-system into the overall product. You have proven that this integration meets your functional requirements. Congratulations, you have closed your TRL.
When used in combination, a PRINCE2-like project management approach guiding delivery of specific TRL work packets can provide a coherent plan to spend a number of resources on achieving an outcome; iteratively building certainty and ensuring constant progress towards the best possible, achievable solution in the fastest and most efficient way possible.
