Regression Analysis Findings

Our regression analysis shows strong correlations with high performance when organizations effectively:

• Prioritize the backlog
•  Have a clear definition of done created within the team
• Don’t over-emphasize development cost when prioritizing requirements

Impact of Respondents’ Perceptions

Respondents’ perceptions of product development methodologies also showed up in our data. Over 90% of survey respondents are actively involved in the product development process. Those who lack an understanding of the method in use clearly are part of organizations that underperform.  Additionally, those who believe using Agile/Scrum leads to high product profitability are more likely to be in organizations that perform well financially. Knowledge clearly makes a difference.

Top Tools and Automation

Lastly, the most highly productive teams believe their effectiveness would increase through the use of better tools and automation. Here, again, knowledge makes a difference. Teams that have implemented effective team processes are looking to expand their uptake of DevOps and Extreme Programming (XP) practices with the latest automated refactoring, test, build, and infrastructure-deployment tools.

Looking Ahead

As of this post, we conclude our exploration of the findings drawn from the latest Global Study of Product Team Performance. Starting next week, we’ll switch gears and begin a discussion of product management tools that can grease the wheel of success within your team and your organization.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Product Development Findings appeared first on Actuation Consulting.

This post discusses the sixth and final key finding gathered from the analysis of the latest Global Study of Product Team Performance.

#6 Key Performance Indicator

Product Teams that Believe Their Effectiveness Would Be Improved by the Use of Product Tools and Automation Are Likely to Be High Performance Teams in Companies that Achieve Their Financial Goals and Objectives.

Just over 50% of our survey respondents are members of technology development teams. For this reason, this finding is particularly meaningful to technology development organizations.

Desire to Improve Tools and Increase Automation

The sixth indicator of high performance is essentially possessing the desire for improved tools and increased automation. Organizations that have implemented effective team processes often seek out ways to further improve efficiency.

This indicator points to the rapid uptake of DevOps and the Extreme Programming (XP) practices that underpin it.

This particularly points to:

• Test automation of every kind starting from the practice of test driven development
• Refactoring (and tools that automate refactoring)
  • Continuous integration, continuous delivery, and continuous deployment (and
  the automated build tools and application release tools that support these
  practices)

The up swell of enthusiasm for test-and-build automation has driven strong tool development and rapid tool advancement. This includes having check-ins kick off build scripts that not only compile binaries, but also generate documentation, tests, and statistics. They also kick off test automation, plus generate and deploy distribution media, website pages, and program logic to servers.

In addition, on the operations side, tooling like continuous configuration automation enables automated rollout of both physical and virtual infrastructure. The result is that teams with effective team processes find themselves continuously looking with longing for the latest – and the latest is rapidly evolving.

Next Post: Wrapping It All Up

For several weeks now we have delved into the responses and analysis of the latest Global Study of Product Team Performance. Next week, we reach the conclusion when I will share a few final thoughts on this interesting survey and its outcomes.

Recap of the Six Key Performance Indicators:

1. High performing teams have a clear definition of “done”.
2. Respondents unable to associate a product development methodology with product profitability are unlikely to be on a high performing team.
3. Respondents who believe using Agile/Scrum leads to high product profitability tend to be in organizations that meet or exceed their financial goals.
4. Teams that consider development cost as a criterion for requirements prioritization are more likely to under-perform (i.e., negatively correlated).
5. There is a strong correlation between an effectively prioritized backlog and high product team performance
6. Product teams that believe their effectiveness would be improved by the use of product tools and automation are likely to be high performance teams in companies that achieve their financial goals and objectives.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Product Tools and Automation appeared first on Actuation Consulting.

Today we will look at the fifth of six total key findings drawn from the analysis of the latest Global Study of Product Team Performance. Next week we will wrap up the full set of analytic findings gained from the latest survey.

#5 Key Performance Indicator

There is a Strong Correlation Between an Effectively Prioritized Backlog and High Product Team Performance

A substantial 43.5% of survey respondents indicate that their team prioritizes the backlog effectively. When we submitted this question to regression analysis, the correlation was clear:

Product teams that effectively prepare and prioritize their backlog of work are more likely to perform at a high level.

Conversely, 37.9% of respondents indicate that their backlog is a jumble. Our regressive analysis shows that product teams that describe their backlog in this way are negatively correlated with high performance. That is, they are unlikely to perform well.

Grooming the Backlog

The importance of effective backlog grooming cannot be overstated. Not only does an effectively groomed backlog ensure that teams are always applying themselves to work that customers will highly value, but it also serves to motivate developers.

In fact, just by working on the top backlog item developers know they will positively impact customers more than they would by doing anything else. That motivational factor cannot be overstated.

Next Post: The Sixth and Final Key Performance Indicator

In the final summary post of this series based on findings from the Global Study of Product Team Performance, we will pull the study’s takeaways together and give you the complete scope of takeaways you can use to build a stronger, more effective product team.

Recap of the First Five Key Performance Indicators:

1. High performing teams have a clear definition of “done”.
2. Respondents unable to associate a product development methodology with product profitability are unlikely to be on a high performing team.
3. Respondents who believe using Agile/Scrum leads to high product profitability tend to be in organizations that meet or exceed their financial goals.
4. Teams that consider development cost as a criterion for requirements prioritization are more likely to under-perform (i.e., negatively correlated).
5. There is a strong correlation between an effectively prioritized backlog and high product team performance

The post Effectively Prioritized Backlog and High Performance appeared first on Actuation Consulting.

#4 Key Performance Indicator

Teams that Consider Development Cost as a Criterion for Requirements Prioritization Are More Likely to Under-Perform (i.e., negatively correlated)

54% of respondents stated that they use development cost as a factor in determining which high level requirement they pursue. Unfortunately, our data indicates that organizations doing so are likely to be poor performers.

We believe that product managers must consider development cost in calculating ROI. It should also be used in determining how to deliver the most bang for the buck to customers. However, study data shows that neither high performance teams, nor teams in organizations meeting their financial goals, take development cost into consideration at the time requirements are being prioritized.

When the backlog is effectively prioritized, there is no such negative correlation. Furthermore, there is also no standout positive correlation with any of the other requirements-prioritization criteria we asked about. These include revenue, profitability, customer importance, internal stakeholder influence, risk, and technical considerations (e.g. architecture, stability, scalability).

Break Down Work to Avoid Cost Concerns

It is critical that product managers and owners break down work into small enough chunks to ensure that high cost is not made an excuse for not delivering the highest customer value. The work breakdown can be by stories (Agile), minimum marketable features (MMFs), minimum viable products (MVPs), or experiments (lean-startup). The important point is that dividing the work, and thus the cost of the work, allows teams to deliver the highest value in the shortest time.

Next Post: #5 Key Performance Indicator

Be sure to check out my post next week when we will discuss the fifth key performance indicator drawn from the most recent Global Study of Product Team Performance.

Recap of the First Four Key Performance Indicators:

1. High performing teams have a clear definition of “done”.
2. Respondents unable to associate a product development methodology with product profitability are unlikely to be on a high performing team.
3. Respondents who believe using Agile/Scrum leads to high product profitability tend to be in organizations that meet or exceed their financial goals.
4. Teams that consider development cost as a criterion for requirements prioritization are more likely to under-perform (i.e., negatively correlated).

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post KPI #4: Development Cost and Prioritization appeared first on Actuation Consulting.

#3 Key Performance Indicator

Respondents Who Believe Using Agile Scrum Leads to Higher Product Profitability Tend to Be in Organizations That Meet or Exceed Their Financial Goals.

Only a certain group of survey respondents identified a methodology responsible for making their product more profitable. Those who named Agile Scrum as the source of their improved product profitability are, in fact, typically in companies that meet or exceed their financial goals.

Agile Scrum Delivers

This result is not surprising. Agile Scrum draws on many best practices and assembles them to deliver synergies that newbies often find eye opening. The practices and the synergies are “best practices” because they deliver higher productivity, better quality, and improved products. Each of these outcomes leads to positive financial results.

An Interesting Fact

The survey shows that over 52% of respondents believe that Agile Scrum leads to improved profitability. Yet, only 48% of organizations say they actually use Agile Scrum. The finding that more respondents associate Agile Scrum with product profitability than actually use it is consistent with results from previous years.

Since 43% of teams say they are high performing, the correlations would suggest that those teams using or at least believing in Agile Scrum make up the majority of teams that are high performing.

Next Week: #4 Key Performance Indicator

My next post will reveal the analysis behind the important fourth key performance indicator.

Recap of the First Three Key Performance Indicators:

1. High performing teams have a clear definition of “done”.
2. Respondents unable to associate a product development methodology with product profitability are unlikely to be on a high performing team.
3. Respondents who believe using Agile/Scrum leads to high product profitability tend to be in organizations that meet or exceed their financial goals.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Agile Scrum’s Impact: Key Performance Indicator #3 appeared first on Actuation Consulting.

#2 Key Performance Indicator (KPI)

Respondents Unable to Associate a Product Development Methodology with Product Profitability are Unlikely to be on a High Performing Team.

Analysis of the latest data illustrates an important point: Product team members who cannot project which product development methodology would make their products more profitable are more likely to be in organizations that perform poorly.

We cannot say exactly why this is. However it suggests that lack of exposure to modern methodologies may be to blame. This includes teams that have been locked into Waterfall and have not yet tried Agile. It can also include teams that are basically in chaos and don’t have any coherent methodology.

“Bad Agile”

On the other hand, there’s a lot of “bad Agile” out there. These are methodologies that are Agile in name only, not in practice or in spirit. Such a situation could lead a team to believe that nothing will help.

Regardless of its root, the second key performance indicator is a remarkable conclusion. It is highly significant to be able to identify the connection between how team members do their work and product profitability.

Next Week: #3 Key Performance Indicator

Watch for next week’s blog post when we will discuss the third of six key performance indicators of high performing teams.

Recap of First Key Performance Indicators:

1. High performing teams have a clear definition of “done”.
2. Respondents unable to associate a product development methodology with product profitability are unlikely to be on a high performing team.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Key Performance Indicator: Development Method & Profitability appeared first on Actuation Consulting.

A Unique Study

The survey itself is unique in that since its inception it has received support from a constellation of leading industry associations and market players. These are groups that generally don’t collaborate on such endeavors.

This effort continued our approach to distributing surveys and following up with regression analysis. We sent direct mail invitations to our subscriber database. Our sponsors and promotional partners also sent invitations to their stakeholders to participate in the survey.

After the survey closed, our statistician conducted regression analysis. The consistency enables longitudinal trending of key performance drivers. Additionally, we continued to use both team productivity and organizational financial outcomes as measures of performance in our regression analysis.

#1 Key Performance Indicator:

High Performance Teams Have a Clear Definition of Done.

The first new key performance indicator underscores the importance of having a clear definition of done. It also reveals the value of having team members themselves create that definition.

Our question to survey respondents was: “Who, if anyone, defines done for their teams?” Nearly 30% (29.4%) of responders indicated that the product team as a whole collectively develops its definition of done. That response was followed by management (23.4%), the product manager (19.5%), the product owner (15.6%), “no one – we wing it,” (7.4%), and engineering management (4.7%).

Results of Cross Correlation

We cross-correlated these results with teams that meet or exceed organizational expectations and learned something interesting. The first result that stood out was a distinct negative correlation between winging it and team productivity. That is, teams that don’t define “done” don’t perform well.

Second, our data definitely shows that it matters who creates the done definition. Organizations in which team members themselves create a clear definition of done are more likely to out perform their counterparts. Product teams that have their product owner draft their definition of done perform slightly more effectively than those teams that develop their definition of done collectively.

There was no effectiveness correlation to having a manager outside the team dictate a team’s definition of done. We think the practice of establishing a definition of done within the team makes team members hold themselves more accountable to the done definition and to each other. Any benefit of having management standardize a definition is outweighed by the demotivating effect of management handing it down as an edict.

Clearly, defining done matters. And when a member (the product owner) or all members of the team (collectively) do the defining, teams are likely to deliver at the highest level.

Next week we’ll explore the second key performance indicator extracted from the latest Global Study of Product Team Performance.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Key Performance Indicator: Clear Definition of Done appeared first on Actuation Consulting.

Question: What is the ratio of UX FTEs to product management FTEs in your organization? (Check one.)

Digging Deeper

The highest responding answer was one user experience full-time employee to three product management FTEs (33.1%). This is a high percentage of user experience employees for sure. One UX FTE to every five product management FTEs received 16.5% of responses. Even combining one UX FTE to more than 20 product management FTEs (15.0%), one UX FTE to 10 product management FTEs (9.1%), and one UX FTE to 20 product management FTEs (3.5%) (27.6% in aggregate), organizations rich in UX FTEs (33.1%) were greater. Clearly, organizations with a higher percent of UX FTEs outnumber those with fewer.

The Culmination

Over the past two months we’ve taken an in-depth look at the results of the Global Study of Product Team Performance survey. We’ve reached the last question. Now it’s time to pull it all together into benchmarks you can use to become a high-performing team. Watch for my next post when we delve into the Six New Key Performance Indicators revealed through the study.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post UX to Product Management Employee Staffing Ratios appeared first on Actuation Consulting.

Question: Where do you think is the most effective place for the user experience team to report?

Insights Gained

Last week we looked at responses revealing where the UX team actually reports in our respondents’ organization. This week we get the chance to learn where the same respondents think the team should report. A little over a third (34.0%) indicated Product Management as the preferred place for user experience to report. That’s only about 3% points higher (30.8%) than what the respondents told us was actually happening. 28.1% consider the CPO as the best person to interface with user experience – almost triple what is actually occurring (10.7%). Only 18.1% would have UX report to Engineering, Development, or Technology, compared to the 30% that actually do manage these reports.

Question: How many individuals are in your core user experience design organization?

Exploring Answers

Respondents’ answers to this question were fairly evenly divided. Interestingly, 23.3% of respondents indicate they have no user experience team. The respondents’ highest response (25.7%) went to user experience design teams with two to three members. 19.4% of respondents said they have six or more user experience team members. 18.6% indicated they have just one user experience professional. The least selected survey response, four to five members, garnered only about half the responses of the highest responding answer (13.0% vs. 25.7%).

The Final Question – Next Week

My next post will consider the responses to the final question on the most recent Global Study of Product Team Performance survey. After that we will pull the survey finding together to reveal the six new key performance indicators shared by high performing teams.

 Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post User Experience (UX) Staffing and Effectiveness appeared first on Actuation Consulting.

• When to share the tactical product roadmap with customers and customer-facing employees
• The reporting structure of the user experience group

Let’s take a look at survey responses.

Question: How far into the future does your tactical product roadmap illustrate future development activities when shared with customers! (Check one.)

Delving Into Product Roadmap Responses

Approaches to sharing product information with customer-facing employees and customers vary. That’s especially true when it comes to how far out they are willing to share. The highest percentage of respondents (24.3%) indicated that their companies were willing to share information six months out. Closely following that are 12 months out with 20.6% of responses and three months out at 20.2%. The answer garnering the lowest response was one month out at 5.2%. Nearly as low (9.9%) was the response indicating that a company shares the product roadmap more than 12 months out. Nearly 20% of respondents indicated that their organizations do not share their product roadmap with customers as all.

Question: User experience is becoming increasingly important to the design and development of successful products. Where does user experience report in your organization?

Drilling Down

Two survey answers received the highest percentage of responses to this question. Almost equal in number, each garnering almost a third of responses, reveal that most user experience teams report to either Product Management (30.8%) or to Engineering, Development, Technology or a similar segment (30.0%). Technology Architecture, Product Architecture, or similar areas and the Chief Product Officer each received just 10.7% of responses. Only 6.7% of respondents named Marketing as the department to which the User Experience team reports.

Coming Up

The Global Study of Product Team Performance provides a treasure trove of insight into how organizations operate and what distinguishes the most productive teams from those that are performing less successfully.

Next week we’ll consider what is the most effective place for the user experience team to report and the most effective size of a core user experience design group.

Advancing the Profession of Product Management
website I consulting I training I toolkits I books I blog I twitter

The post Sharing the Product Roadmap with Customers appeared first on Actuation Consulting.