We joined forces last week with our pals at ProductPlan and Notion—collectively known as The Product Stack—for a webinar called, “How Product Managers and Agile Dev Teams Can See Eye to Eye.” Check out the video and the raw transcript below, stay tuned for more Product Stack events coming soon, and join the conversation on Slack.
How Product Managers and Agile Dev Teams Can See Eye to Eye from Pivotal Tracker on YouTube.
Andre Theus: Welcome to today’s webinar, “How Product Managers and Agile Development Teams Can See Eye to Eye.” My name is Andre Theus and I will be your moderator for today. Today’s webinar is brought to you by The Product Stack. We are a coalition of product managers helping you to plan, execute, and measure your way to customer-centric products. This coalition is backed by three companies: ProductPlan, Pivotal Tracker, and Notion.
Today, we will be hearing from three very experienced product management veterans. First, we will hear from Kevin Steigerwald, who is a cofounder and chief product officer of Notion. He’s a full-stack product designer with over 10 years’ of experience designing, researching, marketing, planning, and building products. From package design in big-box retailers to indie games in the App Store, his passion is for designing experiences that make our lives easier and more enjoyable. He believes pies are for dessert and not for charts, tortillas make the best utensil, and the iPhone 4 is the proper screen size for a phone. Welcome, Kevin.
Next up is Dan Podsedly. Dan is a Pivotal Labs VP and the general manager of Pivotal Tracker, the popular project collaboration tool for modern software teams. Dan has been with Pivotal Labs since the nascent days in the mid-2000s and has experience in every aspect of software development, including engineering, product management, and UX design. Welcome, Dan.
Next up is Jim Semick. Jim is a the cofounder and chief strategist of ProductPlan. For over 15 years, he has helped launch new products now generating hundreds of millions in revenue. He was part of the founding team at AppFolio, a vertical SaaS company. Prior to AppFolio, Jim validated and helped launch GoToMyPC, GoToMeeting. Both of them were successfully acquired by Citrix. Jim is a frequent speaker on product management and the process of discovering successful business models. Welcome, Jim.
Jim Semick: Great. Thanks, Andre. Great to be here.
Andre Theus: Today our speakers will dive into the dynamics of agile development and product management. Agile development sprints and long-term strategic plans don’t always go hand in hand. Don’t be that guy falling behind. Hopefully our speakers will share some practical advice for product managers that work with agile teams and will help you to sprint better together.
First, we will hear from Kevin, who will talk about the right metrics for agile product managers, and then Dan will share some tips that will help you to work better with your agile development team. And last but not least, we will hear from Jim who will give you five recommendations to be a better agile product manager.
But without further ado, let me hand it over to Kevin who will talk about agile metrics. Kevin.
Kevin Steigerwald: Great. Thanks, Andre. Let’s get started talking about agile metrics. I think the biggest takeaway going into this part of the conversation is going to be, everything you’ll learn here is just strategies of what you can do with your own team. There’s not in my opinion a single metric that’s right or wrong. It’s what you do and how you use it.
Before we get started, let’s do another quick poll. We’re interested in what’s the biggest challenges your team is currently facing in using data to collaborate, so that would be knowing what to track. There’s obviously a million different things that go on day to day with running a development team, so tracking the right things can be hard. Collecting the data is a difficult task, so interested in knowing where these things fall. That can help shape our direction here. It looks like getting a handful of responses which is great. We’ll do a couple more seconds here. Any stragglers coming in.
Great. Yep, pretty even spread there. Curious to know what the others mean there for you guys, so maybe if anyone has specific questions that don’t get addressed here feel free to add those, but “knowing what to track” and “collecting the data I need,” those are definitely two of the biggest problems we see that we talk to with our customers using Notion itself. Let’s go through a couple real world examples, but I think before we really dive in, it’s important to know why does alignment around these metrics matter? Because as you just saw, everyone’s having difficulty knowing what to track, how to track it, so why even go through that pain point? When you’re running multiple teams, and even when you have a single team, a development team is still then just one wheel in your chain.
By tracking the right things, you’re creating buy in on the goals and your team and it becomes easier to then communicate those goals up the chain. You’re a product manager. It’s often your job to be the go-between between the development team and sales or marketing or customer support, and the more alignment your entire team has from the bottom up, it’s easier to communicate those things knowing that you’re not just speaking for the entire team but you have their entire support behind you as well.
Also, one of the biggest important things is it allows for a little more accuracy when you’re planning out your roadmap. If you have the right data in your hands you can more accurately plan that out and then feel confident in your projections. One thing to note. I put it very small but important warning down at the bottom is when you’re tracking metrics for teams, regardless of what type of team you are and what you’re doing, just be very mindful of metrics that are tracking individual performance. I think Dan goes into this as well, but software development is a team sport and individual metrics will set you up for failure.
Let’s get started. Velocity is a very common metric. I’m sure everyone here is familiar with it, but when you’re working with multiple teams velocity could actually be very difficult to use as a way to compare the overall health of your team and in sense then the overall health of your development process and product. Mainly because different teams may use different point sizes, the way that they estimate. Even if they are on the same point sizes, what’s a 2-point or a 5-point or a 10-point to one team might be a completely different size or number for another team, so you might get these very wide ranges of velocities with no real way to compare and say what team needs the most attention, where are we hurting the most? Next slide, Andre.
What should you be tracking instead? Basically you want to use that velocity. You want to use the points that are being delivered, but there’s ways that you can normalize that data across your teams. So if you’re a velocity-driven team in the way that you plan, you can take the points that are being delivered and divide that by the current velocity for each individual team, and what that’s going to give you is a consistent metric where you’re looking at iteration over iteration, are we consistently hitting that same percentile? If there’s a lot of change in that number, then you need to dig in with your team to understand why is there that volatility?
Same thing if you’re capacity driven. You can do points delivered divided by the points committed to at the beginning of an iteration. I think it’s important whenever you’re looking at something like that thinking through, what does committed actually mean? That’s not necessarily an agreement that these will be delivered, but that’s what was the team comfortable with at the beginning of an iteration saying this feels within the world of reason that we’re going to be able to deliver this, and then how well did they actually hit those numbers? Again that’s going to give you a normalized number across the board.
If you look at on the next slide here you can start to see other things to watch out for as you’re tracking this. These graphs at the top here. You have two teams, A and B. You’re tracking this new way of looking at your normalized data across each team. You can see in that sixth iteration, both teams took a dip from their normal capacities and there could be a number of reasons why that happened, but if you’re going to see a dip, seeing it across multiple teams or all of your teams is actually probably a good sign that there’s something that affected everyone, not just a specific team. Whereas with team B on the fourth iteration, they obviously were way off the mark and so that’s again another area for you as a talking point to go in and be able to better understand in your retrospective or just by talking with the team. What happened this sprint? Was someone sick? Were requirements off? Things like that.
The bottom charts there are looking at burndown. It’s another key metric that you want to be tracking along with this. You want the chart on the left. You want this consistent flow of tickets moving through your iteration, whereas a chart on the right where everything’s coming in at the very last day might be a sign that the team might be sandbagging their numbers or just throwing all their tickets to be done to hit that 99, 100% everything got delivered that we said would, as opposed to is it actually consistently being tracked? That’s one example of a better way to approach velocity. Another one that we can look at is throughput. If you could advance the slide, there we go.
The problem with throughput when working with multiple teams is throughput encourages quantity over quality. It doesn’t necessarily account for team size, so if you’re looking at the throughput of a handful of teams, which team here is really doing better? Is better even a measure that you want to be tracking? And the truth is you really don’t have that answer just by looking at that single number, so what do you do instead? Similar to with velocity, there’s a way you can normalize this data. Next slide, Andre.
A better way to track just throughput then would be taking a look at just the work in progress and dividing that by the numbers on each individual team. So if you have three different sprint teams, you can take individually each team how many members they have, then look at their work in progress. Your end goal here is optimizing for a consistent flow to identify repeatable behaviors. Each team is going to be different and it’s important to keep that in mind. You’re not looking for ones across the board here. Certain teams might be working on multiple tickets at a time, multiple stories, but that may be their constant behavior from iteration to iteration. Again, it’s about looking for consistency but by doing this it’s normalizing those patterns as opposed to just looking at something like pure throughput.
Some other things to watch out for. You want to be tracking your iteration flow, so as your stories are moving through the iteration, are you identifying any plateaus? Looking at that chart going from the third, fourth, fifth, sixth day there, not a lot of tickets have been moving and so your work in progress chart isn’t really going to be updating at all, but that may not be signifying that something is really happening. There might be a roadblock, so you want to be tracking that as well. Same with cycle times. You can see what’s the average time stories are spending in the different states of the progress. Might be a great way to identify where exactly bottlenecks are.
Then I think my last slide is just some additional tips and metrics you want to be tracking across the board. Some good tips. Sharing dashboards. Increase your visibility for your entire team. No one wants to know that they’re being tracked but they don’t know what is being tracked. No one wants to know that a manager has this oversight and is measuring things but they themselves can’t help improve those without that visibility. Getting the entire team on the same page is a great way to again get that buy in and increase your visibility, an improvement.
Consistency is super critical. Regardless of what you’re tracking, as long as you’re staying consistent in how you’re tracking it is very important, so that from iteration to iteration, day to day, week to week, when you start to compare those numbers across your teams that there is that consistency and you can trust the data. Because again, you’re looking for trends. You’re looking for movement. You’re not necessarily looking for, “Well, our throughput is 30. We should double that. It should be 60.” Those are arbitrary numbers that will change throughout projects and things you’re working on.
Other metrics you want to look at. Defect removal efficiency. How many escaped bugs are being found in production versus in staging? Again, that’s placing emphasis on the quality of the work, not the quantity. Things like story churn is a way to identify if things are again being blocked higher up in the process that maybe a debt metric wouldn’t specifically track, so that might be a problem with the requirements that the product is surfacing, and then something like team happiness. Poll your team. How happy are they? Are they pleased with the work they’re doing? Are they excited about the upcoming work? Things like that will start to give you this qualitative data that will help you balance out quantitative data, and then in the end again giving you that complete picture across all of your teams to stay as efficient and agile as possible.
Andre Theus: Right on. Thanks so much, Kevin. Hope all of you are now armed with the right metrics to track and know what perhaps you shouldn’t track. Next up is Dan. If you have the metrics in mind, now Dan will give you some recommendations on how to efficiently work with agile development teams. Before I hand it over to Dan, quick reminder. We see lots of questions coming in. Keep those coming. In my opinion the most important portion of a live webinar is that you can get your questions answered after the presentation is done. Dan, do you want to take it away?
Dan Podsedly: Will do. Thank you, Andre. Kevin, it was a great presentation. My purpose here is to talk a little bit about how the product manager, how we can work best or most effectively with our development teams, and hopefully by doing that effectively we can align some of those metrics and keep them really consistent, which I agree is the overarching goal. As always, before we jump into this section there’s a quick poll. Thanks, Andre, for bringing that up. We’re curious about where you run into the most headwinds. Where do you find the most trouble? I know these are not the only areas that are involved in getting features out the door, but these are some of the bigger ones so we’re curious. Is it design? Is it to do with development, technical issues? Do you have a lot of dependencies between stories? Is it testing? Or is it something else or perhaps it just works well for you. We’ll take a few seconds here.
Great. Thanks for all the votes. It seems pretty evenly spread out. I think the overall theme here is that developing software is hard and getting those stories to the finish line has a lot of challenges but it looks like a little bit more so on stories that have some dependencies and maybe just how do you get them tested and acceptance can be challenging, so I’ll try and touch on some of those in the presentation.
Let’s jump right in. First slide. First, a little bit of framing. When we talk about our role as product managers and even just what the role is, it’s evolved over the years. It used to be more of a business analyst role where we wrote a lot of requirements, then we handed it off to remote or faraway teams, and I think as an industry we’re trying to move more toward the ideal which is working as a team, working as part of a team. The PM as the center of communication. You’re part of the team. You’re there to frame what we’re trying to solve, why. You’re managing the timeline, priorities. You are the conduit of communication, and it’s worth noting what cross-functional means in this context. It means all of the roles necessary to get the job done to solve hopefully a customer or product problem working together rather than handing work or passing work from one department or one group to another to another to another.
Linear workflows are great for manufacturing. There are some inherent constraints in those environments, but software it’s all about the communication, and so we need to come together. We need to work together to get the job done. This is the best way to interact with teams, to be part of the team, to be the team essentially. Next slide, please.
You have your roadmap and I think Jim’s going to talk a little bit about how you manage that roadmap, and hopefully that roadmap is more of a path or a prioritized list of problems to solve, but let’s say we’ve taken the next thing on the roadmap and now we’re starting to explore how to solve the problem, and that typically involves design. Pretty much anything user facing, even if it’s an API, should have thoughtful design before we just jump in and start implementing it.
A common pitfall for product managers in my experience is that we tend to have the solutions in mind right away. We see a problem, we imagine a solution. I think we’re all guilty of this and this is I think one of the hardest things to transition away from. Your designers don’t want you to bring solutions to the table. They want you to bring your problems. Your problems framed as problems, and perhaps along with the problem you have some hypotheses about how you might be able to solve the problem, but it’s more about the problem and it’s about the desired outcome, the product outcome, the business outcome. You are working with your designers to come up with solutions, but that’s more on them. They are the ones who have the skills, the training, the experience to figure out how to solve the problem. What’s the right interaction? What’s the right feature? How should that feature work?
You are not handing the problem over to them and walking away. You are in the room. In fact, it’s actually great to have everyone or at least all of the roles that will be working as part of your team in as much of a subset as you can to be in the design conversation, to be in that room. It doesn’t mean you’re there all day everyday, but you can facilitate, for example, workshops that bring a couple of developers, a tester, perhaps other roles, even a potential customer or user into the room to generate ideas. There’s all kinds of techniques for brainstorming coming up with ideas. Honing in on the right or the most promising solutions, and then your job is also to help validate what you’ve designed through prototypes, through customer interviews, etc., so you are very hands-on but you’re not solving the problem. You are not the feature designer.
Finally, one thing that we see a lot, or one of the challenges we see when we try and hire a product manager in the interviewing process is this issue of not being able to kill your darlings, and it goes back to the original point here, which is don’t come with solutions, but people really do have a hard time letting go of their ideas, and I think the best product managers are very open to letting go, to letting objective information, objective data, research from your customers or with your customers, metrics even, drive your product decisions. That’s an important part of this. Next slide, please.
Let’s say we’ve designed something—and hopefully it’s a feature—but it’s not one that’s going to take years to implement. We’ve come up with an increment of a solution that heads towards a milestone, that milestone being a learning and validation opportunity. We want to ship something but something that we can get done in a matter of weeks ideally. So now what? Now what do we do with this? Well, the heart and the unit of progress and collaboration in software development is hopefully a fine-grained user story. That’s what we believe. That’s what some of our tools are designed to help you manage. We’re breaking down the problem or the feature we’ve designed into small stories.
What’s a good story? A good story, it’s vertical, meaning that it’s not half of some piece of functionality. When you finish that story, when you check it in, the system is in a good place. You can actually use it. You can do something with it. You can verify it. We’re just making these incremental steps toward that milestone which is shipping that feature, getting some validation for example, and these stories should be small. We’re talking about days at the most for each story, not weeks. When you’re talking about week, two-week-long stories, what happens? You learn a lot. You discover a lot by working that story to the finish line. You’re 80, 90% done. Then you realize there’s some assumptions that were made. Big stories have a lot of complexity, a lot of risk, and they just take too long so break them down to small stories. A couple days each.
Stories should be more about the intent and the outcome rather than technical instructions. For example, some of what we have in the lower left hand corner here. Stories shouldn’t be telling developers to add a particular column to a database table. It’s about what is ideal, what’s the outcome from a user or from a customer perspective? Stories shouldn’t be big. It should be something that any pair of developers can pick up and they can execute on it. Maybe they ask a few questions. What we have on the right is a better example of stories that follow these guidelines.
Finally, the last question that comes up a lot is how far out do we plan in the backlog? Your backlog is not your roadmap. The roadmap is what gives you a path along a current path. The backlog is an execution. It’s tactical. It’s focused on the team getting to the next milestone or two. From our experience, two, three weeks, maybe a month worth of nicely broken down, groomed, estimated stories is great and everything we do is about maintaining that two, three weeks’ of backlog and keeping us focused, and your job as a product manager is to stay on top of that. You’re feeding a team with these stories. If they go too far out, it’s really hard to manage, and things are very likely to change if they’re more than let’s say a month, six weeks out from now. Next slide, please.
Now we’re developing. We have our nice, crisp backlog. It goes out maybe four weeks. How do we really work with the team? First of all, we have to work with the team. We don’t just hand that backlog and those stories to a bunch of developers, walk away, and say, “Hey, let me know when you guys are done. I’ll come back and then we can do a demo day or something.” You need to be involved at the level of every story, and all of these things that we do, because we’re doing some sort of an agile process whether it’s XP or scrum. There’s all these ceremonies like standups, planning meetings where we go over the backlog. We break things down. We groom. We estimate. All of these things.
Really, it’s all about communication. It’s about getting everybody on the same page. It’s about alignment. It’s about socializing. It’s about staying focused on the most important things, so these are all team activities. In a lot of situations that we see, the PM and maybe the engineering lead or the architect, they are the ones who are breaking stories down. They’re the ones who are estimating them. Then they’re just presenting to the team, “Here you go. These are your stories. We’ve already presized them for you.” That’s missing on all the important conversation, on the socialization, so we want to do that as a team.
Because communication is so important here, and because we don’t want to have our stories be so dense where we’ve spelled out every little thing and we’ve had to think of every little detail, it’s important that…and it’s common to refer to stories as evolving conversations, but they cannot evolve if you haven’t organized and optimized your communication process, your tools, and your availability. As a PM, one of the most important things is to be available, to be able to respond to questions as they come up on stories immediately, so what is your team using for communication? Is it just email? Are you practicing Inbox Zero or maybe just Inbox 50? You can’t have a mess in your email because you’re not available. People can’t get hold of you. Same thing with Slack. There’s all these great tools out there that you can use, and as long as you configure them, keep them organized, have push notifications enabled and make sure that your priority is to keep progress moving by being responsive, being available by answering questions, responding as needed.
Then finally, Kevin mentioned consistency as an important theme in metrics. It’s equally important in your management and planning of the backlog and what your team works on. It’s common to focus really hard on getting that big feature out and all you’re doing is just cranking out features, only to hit this wall of bugs or, “Oh my god, now we have all this technical debt, we have a lot of refactoring to do,” and then for the next month you’re doing nothing but…you’ve gone dark. You’re doing a bunch of backend work. You’re cleaning up the code, but your customers are not really seeing anything and your team gets stressed out because you’re swinging from this one extreme to the other.
I think the best approach is consistency. Try and maintain some slack in your timeline so you have room for the unplanned things that come up and you’re not going to be able to predict every little thing. We found that as we build features, we start with maybe a third of the stories that will actually come into play and get prioritized by the time that feature is complete. Two-thirds is emergent planning, and you have to be able to consume that as you go, and so you need a balance. That which you planned. That which will emerge as you get stories out, as you get feedback, as you learn from that technical debt and inbox. Just keep that steady every week. Next slide, please.
Finally, OK, we’ve gotten a bunch of stories finished. We’ve all aligned around what needs to be done. Great. Now we see a bunch of stories have been finished and delivered. Now it’s your job as a product manager to get those to the finish line. The finish line is accepted, or whatever the term for that is in your project management tool, but what’s that final state where we verified it, we’ve completed the feedback loop? A lot of times the conversation is about testing. How do we do testing? What’s my role as a product manager? And sometimes we just hand these stories over to our QA department. But then guess what happens. Days back and forth, maybe sometimes weeks. They don’t have the context. They’re filing bugs. You’re like, “Those are not bugs. That’s on purpose.” And you’re just spending all this time with the back and forth, so first of all get rid of your QA department.
It doesn’t mean go fire all those people. Make them part of your team. That should be part of your cross-functional team. These are testers who have a specialized skill when it comes to exploration, digging deep. You just bring them into the story conversation and acceptance as needed, but you are the chief accepter on every story. That’s your job as a product manager, and it’s not just testers.
Designers should be involved in making sure that stories get implemented as per the design intended, and there’s other roles. Sometimes you have developers who are really good at exploring, and there’s always the question of do we do all this testing at the level of each story, or do we do a more holistic approach? We’re big fans of doing just enough testing at the level of individual stories just to get them accepted and have that quick feedback loop, but then you plan and organize test charters to allow you to do broader exploration and make sure the feature as a whole works, or a very focused one. Security, cross-browser testing, et cetera.
Finally, last point, beware of the Christmas tree. What does that mean? This is a very Tracker-specific thing and this is something that our CEO is very fond of going around and nagging people about, and that is stories that are waiting for acceptance for too long. When that happens you have this backlog, or you have a list of stories and you see the Reject and Accept buttons, so lots of red and green. That’s why it’s a Christmas tree, and Rob our CEO is very keen on this being the worst thing you can do as a product manager. If you let those stories sit and if acceptance takes too long, everything breaks down. Your whole communication process breaks down. People have to switch context. Things just get completely out of flow and it’s a big problem, so I would focus on that. Get those stories accepted quickly. Make yourself available so you can do that and I think things will be in a better place. I think that’s all I have. Thank you.
Andre Theus: Thank you, Dan. Hope everybody got some good recommendations on how to work with a development team. Last but not least we will hear from Jim and Jim will share some challenges as well as some recommendations on how to be an agile product manager. Jim, you want to take it away?
Jim Semick: Yeah, let’s do that. I’ll talk about some of the challenges as Andre said, but first let’s do a poll. I want to hear a little bit about which stakeholders are involved in your roadmap planning process. It seems to differ per organization, so let’s take a few minutes here or a few seconds and like to hear who the stakeholders are, and you can select more than one. OK, terrific. As expected, product management, executives are the top stakeholders. It’s always surprising to me how many stakeholders have input on the roadmap from sales and marketing, and not that that’s a bad thing. Obviously selling to customers and prospects is key, but I’ve talked to a number of organizations where sales and marketing is really the loud voice that makes things happen on the roadmap. We can talk a little bit about that. Let’s go ahead and talk first about some of the challenges with being a product manager in an agile environment.
One of the first challenges is to stay flexible while still driving that long term strategy, and that’s a real balancing act for a lot of product managers because we’re responsible for that long term vision, where the product is headed over the next year or two, yet at the same time we’re operating in this environment where priorities change consistently, and so that’s a key challenge. Another challenge is to strike a balance between this idea of the traditional product manager being the driver of the roadmap and providing direction to the team yet allowing the team to be self organizing, because that’s a real hallmark of agile is allowing the teams to be self organizing, allowing them to have input into the solutions that they’re creating, yet as a product manager we are focused very much on providing direction so that’s a real balancing act that we have to have.
Another challenge is trying to avoid making commitments to stakeholders six months ahead of time about what’s coming out in Q2 and Q3 and being agile, but the reality of most organizations is that you do need to provide a plan to these stakeholders. You need to give them some inkling of where you’re headed, especially if they’re stakeholders that rely on product release dates such as the marketing team. Another challenge is as product managers, we’ve been there. We’re day to day. We’re fighting fires. We’re accepting stories. We’re managing that backlog. Yet there are these important things we need to accomplish, like long term strategic planning, engaging with customers, and I think all of us struggle with that idea of trying to be a long term visionary, long term product manager, thinking ahead about problems, yet fighting the day to day fires.
Finally, there is always managing expectations. Managing your expectations of what is coming out of the team. Managing stakeholder expectations, and that is both the engineering team as well as other stakeholders, and so these are real challenges and so I have a few tips for working with these challenges. A lot of this echo what Kevin and Dan were talking about. The first tip that I have is to be a broken record, and what I mean by that is to bring it up a level consistently. To evangelize the strategic goals, evangelize the why behind what you’re doing. Talk about problems that you’re solving for customers and for the marketplace. Talk about the big rocks. Rather than getting into these planning meetings and instantly jumping to the story level, or even at the epic level. Repeatedly bring it up a level within the planning meetings.
Why are we doing this in the first place? In these grooming meetings I think it’s essential that we’re talking about real customer problems, and the more that you can bring to the table with evidence from customers about the problem that you’re solving I think it’s going to make your planning meetings go so much better, and the same thing holds true when you’re talking with stakeholders whether it’s the marketing team or sales to always bring it up a level and talk about the why about what you’re doing.
The next tip I have for you is to create a shared understanding of what you’re trying to accomplish, and you can do this a number of different ways. One of them is to share consistently customer feedback, and here at ProductPlan this is something that we do consistently. Everyone in the organization sees product feedback. Even some of these larger organizations that I’ve been a part of that have hundreds of employees. That customer feedback and customer problems are viewed by everyone in the organization, and I think that’s really key. Some might feel that that’s noise and overhead but I feel that that is the reason we’re here is to solve these problems, so any sort of evidence that you can provide whether it’s customer feedback, customer quotes that you can give, any sort of insights into net promoter score and other customer feedback I think are essential to getting everybody rowing in the same direction so that when it comes time to be in those spring planning meetings, those grooming meetings and so on, people understand the why.
A technique that I’ve liked to use in the past is when you’re validating new features and new services, it’s really a good idea to invite particular team members, engineers into those customer interviews so they can hear the voice of the customer. I’ve seen time and time again when you do that, these engineers will come back to the team and they’ll disseminate that information and they’ll create for you that shared understanding. I think that it’s much more persuasive to have somebody else participating in those meetings and to bring that back to the team than for you to do it. Another thing you can do with stakeholders is to continually review prototypes. Talk about the persona, and I’m talking about not just the prototypes that are part of the stories. I’m talking about some of these early stage prototypes, the direction that we’re going in. I think it’s really important to help create that shared understanding.
Then there are team activities. Dan talked quite a bit about you being part of a team, conducting team activities. Activities like story mapping I think are really a great way to increase ownership, team participation, engagement, and best of all they’re a lot of fun so I think that activities like that create a better shared understanding of what you’re building for your customers.
Another tip that I have is that for a lot of folks that have evolved from a waterfall environment to more of an agile environment. When you’re moving to agile there might have been a belief that well, I can take a step back because my team is doing a lot of the work. Or, I don’t need to think ahead as much because I’m no longer writing these exhaustive PRDs, product requirement documents, but actually the opposite is true. You’re part of a product development team and as Dan said you need to be participating in these planning meetings, grooming, and demos, and then actively accepting these stories as they come up.
You really need to make yourself available. As compared to this old waterfall method where we write these exhaustive specifications and product requirement documents, this is actually harder in a sense so I think just accepting that and knowing that you can’t be everything to everyone, and I think this is a trap that a lot of product managers fall into is that they think that they are and they need to be that single source of truth for all questions. But I think it’s important to trust the engineering team, to trust the designers, to trust UX and give them a voice and delegate.
My next tip is to think in themes and goals when we’re engaging with stakeholders. On the roadmap, rather than the roadmap being a list of features or even epics, prioritizing them and presenting them in terms of themes I think is a really important way to do it, and what I mean by a theme is to put what you’re trying to accomplish in terms of customer value. In this example here we have customers need to complete their first purchase faster, and there might be some metrics tied to that to know whether we’re successful in achieving this goal, but then the features that go with it and the epics that go with it fall underneath this theme, so when you’re talking with stakeholders if you can put it in terms of what the customer is going to achieve out of it, you’ll do a lot better in getting stakeholder engagement.
Finally my last tip is to foster trust through transparency. What I mean by that is to make your entire product management process, including the roadmap and the way that you prioritize the roadmap, make that an open conversation. Especially with stakeholders. I think this is really important to give stakeholders self service access to the roadmap, whether it’s through if you’re using Confluence or SharePoint or some other tool to make the roadmap accessible, and if it’s a living roadmap that is constantly updated than half your battle is done, because your stakeholders are now on the same page. Then to talk about how you prioritize the roadmap I think is really important as well. To talk about the method that you use, your thought process into why you’re doing this versus that I think is really important.
Then, sharing in the glory. Showing off the teams’ work in demonstrations and then making sure the stakeholders are not surprised by what’s happening on the roadmap. Making it an open, constant conversation I think is really important. Those are some of my tips, and hopefully we’ve left enough time for some questions.
Andre Theus: Thanks Jim. Hope everybody got some good takeaways out of those. Now we’re moving on to what in my opinion is my favorite section of a live webinar. All of you attending live and the benefit of attending live is that you can ask questions, and we’ve seen a lot of questions coming in so thank you everyone who submitted a question. We’re going to do questions for about 10 to 15 minutes so please keep them coming. We monitor them and we try to address as many as possible, so keep those questions coming. The first question. Kevin, why don’t you take this one? The question is, “With shifting priorities from sprint to sprint, what are ways to report a projected deadline with confidence?”
Kevin Steigerwald: I’ve been answering similar questions to this in the chat as well. It’s tricky. I think everything that Jim was just talking about plays into this. You need transparency into the entire process. That’s definitely going to aid you in these scenarios. It’s also another common theme that’s being popped up throughout. The questions being asked are around how do we do better estimations, or my team doesn’t estimate at all. How do I prove the value of it? This plays into that too, and so step one, getting everyone on that same page and understanding why it’s important, and why it’s important is so that as a team, as a company, you are delivering your product to customers in an efficient way. By having things broken down as minute as possible ahead of time, it allows you as those priorities shift, you should be able to better judge what the overall effect that has on any projected deadline.
Your confidence is going to come through both in knowing those things, but then being able to look at those historical data points, so by tracking your velocities, your throughputs, normalizing that data with some of the examples just shown, looking for other scenarios where maybe this happened in the past. If some priority comes in last minute to your current iteration, and maybe that’s going to completely sideline all the work that you’ve just been doing, well maybe that same thing happened six months ago. Maybe you can point to that and say, “Look, the last time this happened our delivery date was pushed back by three weeks. It’s very possible that this is going to happen again.” Look into your data that way and then making the entire process as transparent as possible I think will help with both of those scenarios.
Andre Theus: Right on. Thanks, Kevin.
Kevin Steigerwald: Yep.
Andre Theus: Dan, the next one is probably a good one for you since I know Pivotal has many offices all over the globe. The question is, “Any advice on working with an agile development team as a product manager in a different location and time zone?”
Dan Podsedly: It’s a good question. It comes up often. The world is distributed these days, and there is one short answer and that is, don’t. Again, ideally the product manager is aligned with the team and as we talked about earlier, being available and responsive is a key part of being an effective product manager working with a development team, and so some things that tend to work well and I think a lot of us are in remote PM positions, but to the extent that we can align time zones and actually work the time zone of the development team, having the right communication tools in place as we discussed earlier like Slack. It has to be real-time communication. You have to have presence, and presence doesn’t necessarily have to be physical in the room but there has to be some form of presence where people can see you online. They know they can just ask you a question.
There’s been some examples on our team where we actually do the presence thing with little stuffed animals. Not necessarily for PMs but our testing and support…our head of support. She has a little toy doll version of herself just to remind people that she’s there and you can reach out and you can ask questions, so I think that’s important.
If you can’t be in the same time zone or you can’t be that available in real time, then I think you have to consider whether there’s a role that isn’t being filled on this team, and perhaps you have to hire somebody that is local to the team that actually acts more as the local product owner, and then you and that person have to have a very strong connection and you have to trust him to be able to act on your behalf to make local decisions around stories, acceptance, etc., and then you basically have a conduit information that happens on a different cadence and a little bit more asynchronously with that person.
Andre Theus: Right on. Good advice. Jim, the next one is probably a good one for you. As a product manager, how do you communicate to your development team the driving factors for a feature? Competitive reasons, customer feedback, etc., etc.
Jim Semick: OK, that’s a great question. I like to think in a couple of different ways about the roadmap, which I do present to the dev team. I think it’s really important for them to see that higher level picture, so rather than their view of the product and the direction of the product being the backlog, to actually show them the roadmap. The roadmap should be talking about the why, and at ProductPlan we talk a lot about creating roadmaps with the strategic goals in mind, and for organizations that do longer term planning I think it’s important to pick a handful of strategic goals for the year or maybe for a longer term. A year or maybe two years, and those strategic goals might be competitive. It might be customer related. It might be providing more value to customers in some way.
So if you pick a handful of these goals and you’re talking about them with the engineering team, with the development team and you do that on a consistent basis, I think that’s a great way of communicating the why behind what you’re doing it, and then backing it up with a couple of things. One is the metrics related to achieving that goal, so are we moving in the right direction? Which competitors are we trying to compete with?
I think that’s important, and then the other thing is to write the roadmap in terms of those themes that I talked about earlier. Bringing it up a level and talking about the why in the roadmap. For example, one of the goals might be to become top three in mind share in the competitive space, or a goal might be to reduce churn by a certain amount. If you were talking about those you can write your themes and maybe even your epics related to those themes in terms of customer value, and then your engineering team will follow and I think over time when they start to hear those reiterated, it will become second nature.
Andre Theus: Great, thanks Jim. Kevin, I think the next one is a good one for you. Do you have any advice for this gentleman? Sounds like he’s in an organization where some teams are still in waterfall and some are using agile or he calls it extreme, where the agile team are having problems setting a date where they will be hitting the MVP for a client, so how do you deal with different project management methodologies in one organization?
Kevin Steigerwald: Wow. Different methodologies are tricky because you’re going to deal with a lot of fractured goal setting and fractured delivery on things. I think take the one piece of that out and just look at the agile side of things where depending on how capital A Agile you are, there’s this idea that delivery dates don’t exist. The work is done when it’s done and it ships when it ships, and while that can be easily true and pointed out from the dev side of things, I think you need to holistically look at your entire company goals and there’s always going to be influence from sales or marketing or customer support about getting things delivered by a certain time.
I think what’s important is from the product owner, product manager and then working with the dev team is again creating that visibility down the chain so that everyone understands why does a date matter so much? And understand that from the dev team’s perspective, if a due date is immovable, then the thing that can or needs to move would be what they can deliver, and so as a PM, you need to be flexible in scope. You can’t have both. You can’t have this unlimited scope or a huge amount of scope if the team says we physically cannot produce that much work by this point in time.
I think that’s where a lot of the concern comes from from the dev side of things why they get worried about a date, and so when you have that specific scenario if half your team is working waterfall and you are working against dates in a different way, that mentality is established and so I think what you need to do is work tightly with your team to understand what’s the scope we can cut? How do we break up this work into small enough pieces as possible? Everything that Dan talked about. To be able to deliver an individual story as soon as it’s done where it doesn’t have to require other work to be done as well, and so how do you break all that work down into small pieces that can be instantly actionable, easier to estimate, and then as a PM you can start to pull things out individually when you need.
Andre Theus: Right on. Thanks Kevin. We got so many questions. Thanks so much for submitting all the questions. We will try to get through a few more but we probably won’t be able to answer all the questions. The next one might be good for Dan. The question here is, “With an emphasis on quality work, what are some best practices for a quality manager when working with product, design, engineering?”
Dan Podsedly: Thanks, Andre. Good question. We’ve struggled with that on our teams as well, and it’s more about how does testing as a role, as a function fit into the agile team, so I’ll try and answer in that sense. We went through a progression of how we involved our testing and QA where it was much about testing or QA as a step that’s part of every story. We would wait for testers to test each story, then the product manager would finally get around to it and ultimately accept or reject, or testers could reject stories, but we found that it’s much more effective to treat testers as an extension of the product manager and people who can keep a different perspective on your system. It’s more holistic, more broad.
One common complaint on our teams is that our testers go to too many meetings, but we actually consider that to be something that’s very valuable because they maintain this thread of knowledge between all the things that are happening across different teams, and they can see how it all fits together and so they’re very much empowered then and they have the right context and perspective to do very effective testing in areas that have a lot of risk, but we focus testing on areas of risk. This is where we write charters and we basically organize testing and focus it on some particular area. For example, if we’re releasing some new feature and we’re getting close to that, we may have some charters that focus on different aspects of that feature. Not individual stories, but slices through the system that make sense together.
We also involve testers in helping customers because it gives them a better perspective on what the customers are experiencing, so the testers actually have this unique perspective where they have visibility across everything that’s happening. They understand the customer because they represent them a lot, but there are situations where you just have to do more formal QA as this step at the end, and a lot of times that happens in the mobile world. When you’re building an Android application and now you have to test on 100 different devices. That’s not something that you’re just going to make go away, and so they can own those activities but those also become these focused charters like OK, let’s test this new release of the application on all the different browsers using whatever hardware we have, rather than it’s being done on every single story that you’re managing.
But overall I think testing is a very important part of the process but it’s kind of like we all do testing, and one thing that also works well is if you can make testers pair with developers if you’re in that kind of a situation. That works great as well, but obviously you need the pairing to be there to begin with.
Andre Theus: Right on. Thanks, Dan. Jim, next one is a good one for you. This question is, “We’ve recently switch from sprints to kanban. I’m having trouble keeping my team on track and completing tasks because the urgency seems to be missing. Any advice on adding urgency or helping motivate the agile team without having sprint deadlines?”
Jim Semick: That’s a good question. I’ve worked in both environments and I can understand why there might be a lack of urgency if there aren’t deadlines associated with things, and in kanban of course there aren’t these estimations that are happening so you can see whether the team is on track. To me, the best way of increasing the urgency is to increase the sense of ownership, and the ownership is tied directly to what the customers are achieving out of it, and so if you can bring to the table real customer problems, and I’m talking about to the point of actually showing the customer, showing the face of the customer or recording an interview with a customer, or doing a screen sharing session and saying, “What are you currently doing today and how can we solve that problem?” Bringing that to the team I think is a great way of increasing urgency.
You can almost put almost like an artificial timeline associated with it. Some sort of way of time boxing the amount of effort before you conduct a release. I’ve used kanban for existing products as well as new products, and if you set almost an arbitrary date to drive the team, and I don’t know if this is agile best practices, but I’ve seen that this does work for increasing urgency. The team and you will tend to back into that minimal product that provides value to customers, and you’ll drive towards that release.
I think those are a couple of tips for doing this. This is actually a really great topic of conversation and I think I’d love to hear some other ideas from people on this one as well.
Andre Theus: Dan, Kevin, do you want to comment on this?
Dan Podsedly: Maybe just one thought to follow up on the urgency. There’s also this maybe false perception that you can’t talk about dates, whether you’re doing XP, agile, scrum, kanban. Like, “We’re agile. We just…” But dates are very important as Kevin pointed out and so that is one of the things that you manage to as a product manager. Because you are the one prioritizing stories and working with your team to get a sense of how complex they are. This is a continuous process and so if hitting a date is important, not just because it gives you urgency but because you need to align with some marketing activities or whatever, a conference, then your job is to reduce scope and to manage scope so that you release something and hopefully that something is just enough for what is actually needed to move the needle forward in some regard. So embrace dates but then be very conscious of scope and work with your team to find the right balance.
Jim Semick: Great.
Andre Theus: Thank you. All right, we reached the top of the hour and this concludes today’s webinar. We had lots and lots of people attending. Thank you so much for all the questions. I apologize if we haven’t gotten to your question, and I would like to encourage everyone to join our public Slack account. You can join for free if you go to www.theproductstack.com, and we have hundreds of product managers that help each other on that Slack account and I would highly encourage you to join that public free Slack account. Then last but not least, all of our products have free fully functional trials. If you haven’t checked out ProductPlan, Pivotal Tracker, or Notion before I would highly encourage you to sign up with a free trial and give it a shot. Thanks everyone for joining. This concludes today’s webinar. Goodbye.
Category: Community
