How Much Effort Does It Take to Create a Great User Experience?
By: John S. Rhodes
Published - April 2006
Blending Project Management and User Experience
How much money does it take to move a project from inception to completion? How do you fully consider the total cost of ownership? How do you know how much time to spend on usability and requirements?
These are challenging questions for the most battle hardened project managers. Not surprisingly, most project leaders have a potpourri of answers and solutions. We all have our own project heuristics based on our education, skills, and experiences.
But I still wonder, why don’t we estimate project effort with respect to what our customers will experience? The answer is pretty simple. We don’t have the right tools. To my knowledge, no one has explained how to do the estimate. Further, if it has been explained, I’m going to go out on a limb. The explanation is not simultaneously simple and rigorous.
The purpose of this article is to provide you with a way to measure the level of effort required to successfully complete a project in respect to user experience. This is a powerful merging of project management, user experience, requirements and best practices. And, it is simple enough for a little monkey to use. More accurately, it is simple enough for me to use.
The Nature of Effort and Output
In simplistic terms, to complete a project you need to expend effort. Effort is measured as time, energy, money, or person hours. The idea is that you expend effort and you get results.
Nearly everyone would agree that you want to minimize effort and maximize output. Further, you want to employ the right resources to get the best output. "Spend a dime, get a million dollar product." You might call this return on investment, but that might be a stretch. It doesn’t matter too much. No need to get hung up on the semantics here.
Wrong Universe So Wrong Measurements
So, how do we measure effort? Well, we can measure time spent. We can measure money spent. We measure a lot of things. The problem I have with traditional measurement is that it is internally focused. We are myopic and selfish in our thinking on measuring effort. We seem to think that we are the center of the universe. Similarly, we seem to think that profits come from within our company.
Many companies probably think they somehow include customers in their measurements of effort. But they don’t. I'm stating that companies need to look outside their walls to understand how much effort it will take inside to get the job done for customers.
Grab your Veuve Clicquot flute, take a sip, and think this over for a moment.
Connecting the Dots to Make the Magic Happen
While it would be nice to consider each and every user, we have to think widely. We need to think about customers as a group, whole, or audience. To stay sane, measurements must be defined this way versus measurements for each and every individual.
We need to care about the total number of people using the product. We also care about how many times per day the product will be used as well as how long users will be engaged with the product. It also makes some sense to consider if the product has some lifespan before the next major update or revision. It probably has a life.
To understand these things you need to have metrics. You might have some previous work to help with your estimates. The key point is that you need to project forward based on what you know about users. You must know something. Take advantage of what you know.
Let's briefly recap. You need users per day, how often the product will be used per day, how long it will used for during those times, and how long the product will be in play in the long run. If you multiply these four simple factors, you will have a single number. This number is the numerator in our equation.
Next, we need the denominator of the equation. While the numerator described above is customer focused, the denominator is internally focused. It is focused on the organizational DNA, although not so much on the project or product under consideration.
First, you need to know the investment and dedication of your organization to user experience. You can use any type of number you want but it is critical to use the same type of measurement on all projects or your results will be blown. For example, the value can be time, money, or headcount. It doesn't matter as long as you use that same measurement in all cases.
Next, you need a value that reflects availability and adherence of best practices and processes. You could use CMMI level for example. If your company gets better over time, adjust the measurement. Again, it is up to you but be careful about being consistent from project to project.
Finally, get a measurement indicating the clarity of requirements. You need to know how well you know them. A simple solution here is to use the same requirements gathering process for all projects. Use a checklist. For example, ask the same twelve questions at the start of every project and get a value for each question. This will keep you consistent from project to project.
Let's recap once again. You need a value that reflects your dedication to user experience, a value indicating your adherence to best practices, and a value indicating the clarity of the requirements. Multiply these numbers together and suddenly you have a denominator for your formula.
In fact, at this point, we can review the simple formula. We have everything we need.
An Example
Let's make sure we're on the same page. Here's the scenario. You know that 100 users are using a similar product twice per day for about five minutes each time it is used. Furthermore, you know that the typical lifecycle with this product before the next upgrade is about two years. Based on a small leap of faith you can easily calculate the numerator:
100 users * 2 times per day * 5 minutes * 2 years = 2000
Now, keep in mind that you would tweak the numbers above to match up other calculations you’ve done like this in the past. Do what works for your organization. You might want to replace the "2 years" with 730 days. I'm advocating more of a mindset than a dogmatic formula.
Let's press forward. Assume that you give your organization's dedication to UX a rating of 7 on a scale of 1 (low) to 10 (high). Also, assume that your organization is CMMI Level 3. Finally, assume that, based on a simple validation checklist, your requirements rating is 4 on a scale of 1 (low) to 10 (high). Here's the simple calculation for the denominator:
UX rating of 7 * CMMI Level 3 * requirements clarity of 3 = 63
Now we have both the numerator (2000) and denominator (63) for the formula. To get the Level of Effort, divide 2000 by 63. If you do this, you get a rounded value of 32. In and of itself, this number doesn't mean very much. I'll readily admit this. But, after just a few projects using the same definitions and standards, you'll start to get a really good feel for the level of effort required on a project to ensure an excellent user experience. The beauty of this approach is its flexibility. You can make it work.
Hopefully you now see how the numerator of the formula captures what your users need, at least in terms of their typical efforts. This is a rough indication of their user experience. The denominator is a rough indication of how well you are able to address the needs of your users through your internal efforts. The final value you derive includes internal and external effort, all with a relationship to user experience.
Way Forward
Even if you don't buy into the formula outlined above, you should still consider the principles I've described. Namely, it is possible to systematically approach the level of effort you need to expend in order to satisfy customers and create a great user experience. Metrics can guide you. The formula is simply a way to align customer UX factors with your organization's UX factors.
For what it is worth, I am convinced that there is a better formula waiting to be born. I'd be happy to have someone come along and blow mine out of the water. That’s fine, but until that happens, let’s use what we’ve got.