Finding the Flow in Breaking Down Work
How “Continuous Refinement” has enabled us to focus on Flow and get more Engaged Developers
How “Continuous Refinement” has enabled us to focus on Flow and get more Engaged Developers
Like many teams, we used to have a “refinement meeting” once a week. However, we soon noticed that this wasn’t ideal and started experimenting with other ways. Over several months, a new process of breaking down work has emerged, where we involve our team from problem description to breaking it down into the smallest units of value.
We are Lorenzo (Product Owner), and Benji (Scrum Master), and have been experimenting with various ways of working for years together with our teams at Hitachi Energy. If you want to learn more, feel free to reach out to Lorenzo or me.
Read on to learn how we break down Epics, who creates work items in our backlog, and how all of this changed the engagement of the team.
Refining our Refinement
We’ve experimented a lot with different approaches on how we break down bigger chunks of work and how we refine smaller items. This has been an ongoing activity for many months. And while we very well keep on changing our approach, we’ve noticed that we are quite happy with what we’re doing now.
What we describe in this post is the current state (as of February 2024), but as the picture below shows, this is a continuous process and most likely not the end of it. And while we encourage you to draw inspiration from what we’ve tried, we also feel that you should challenge what you’re doing and look for better ways within your context.
Context
To make it easier for the readers, we want to give some context. The team where this process is in effect is working on a single product. The team consists of 7 Developers, one Product Owner, and a Scrum Master. The team has been in this configuration for the past months, with the last big change happening 9 months ago with several team members leaving to focus on a different product.
We use the Scrum definition of Developer throughout this post, meaning all team members excluding Scrum Master and Product Owner. That doesn’t mean all Developers have the same skills, we do have different roles like Frontend engineers or QA specialists.
We work with “Epics” that are broken down into “User Stories” that are smaller units of value. The Epics are the “smallest units of value” that make sense to independently release in our environment, while the stories are supposed to be the smallest units that have some kind of user value. We use the term “User Story” here because our work tracking system calls it like that. They are, in most cases, not actual stories from users. In many cases, the real stories from users are captured in the Epics.
From Cadence to Continuous
Over the past months, we’ve been slowly moving away from the Scrum Events that happened on a cadence. Recently, we found that Sprint Planning no longer serves us, so we’ve eliminated it. We still hold a review every two weeks, but now our focus is on continuous flow and making decisions just in time.
You don’t need a Retrospective to Inspect and Adapt your Effectiveness
Most teams that I have worked with run retrospectives. Usually, they are held at the end of their iteration. After…medium.com
We’ve moved away from the Scrum Framework, not because it’s ineffective, but because it no longer aligns with our environment. Gradually, we shifted our focus toward flow, as the fixed cadence of Scrum Events started to hamper rather than support our progress.
Problems with our Refinement
When we talk about refinement, we mean everything that helps us get more clarity on our problems:
Product Backlog refinement is the act of breaking down and further defining Product Backlog items into smaller more precise items — The Scrum Guide
This can be done on individual stories (for example how we technically implement them) or for completely new functionality where we want to find out what we can do to solve those problems.
While we went through our journey of how to work, we encountered several problems:
Firstly, they tended to be one-sided, with the Product Owner explaining Epics to the team, but with limited involvement from others beyond asking a few questions.
Furthermore, the team lacked involvement in breaking down the work, often leaving the final breakdown into smaller work items to individuals (Lorenzo and me) who ultimately weren’t executing the work. Despite our efforts, we never managed to achieve true buy-in from the team on the broken-down items, but rather a mere acknowledgment of their existence.
While the team was involved in discussions about how to solve a problem, the final breakdown into small work items was done by people ultimately not doing the work (yours truly and the Product Owner). While we had good intentions, we caused more issues in the name of “efficiency” than we helped.
Relying on a single person (Product Owner) to add all items to our backlog and “preparing them” for the team, creates a bottleneck.
We found it more and more cumbersome to find a problem that precisely fits in a Sprint timebox, rather than simply focusing on the next most important slice of work, independent of how long it might take.
Trigger to Change
At one point, the Epic we worked on was not progressing as we’d hoped, we did an ad-hoc retrospective to find out what might be the cause. The Developers pointed out that we were part of the problem. With the root cause being us having broken down the work in a way that caused lots of dependencies. Big kudos to the team for speaking up, as this was the trigger to co-create a new approach. One that addresses the above mentioned problems, and ultimately led to what we describe in this post.
Epics — Problems to Solve
Our Epics are not detailed plans on what to implement, instead they are problem statements. To keep focus, we only want one of those Epics in progress at any time. But while the team actively works on one of those, we start the preparation for the next one.
The Product Owner prepares the next Epic by making sure we know what problem we want to solve. It’s important to mention that we don’t look for “Features” with Acceptance Criteria, but that the “What we want to achieve” and “Why” is captured.
On Commitment and Just-in-Time
Once we commit to an Epic, we invest significant time in solving it. Until we proceed, we avoid consuming our team’s time unnecessarily, aligning with a ‘just in time’ approach. The “commitment” happens once we pull an Epic into Analysis. When that happens, we start involving the Developers actively and we’re reasonably sure that this is the next thing we’ll be working on.
This commitment also leads to making sure we’re making those decisions “just-in-time”. We don’t want to have 5 Epics being analyzed, and we want to hold off Analyzing and consuming the teams time till the last responsible moment. That enables agility, as up to this point, we can still pivot on what we want to work next without context switching or losing focus.
Do you wonder how you can forecast release completion before performing an extensive breakdown? Wonder no more:
Using Probabilistic Forecasts for Feature Planning
“We don’t need to plan, we’re agile!” — if you have heard this before, you’re not alone. I’m convinced that by now you…medium.com
Our Current Workflow
Below is a description of our workflow for breaking down an Epic into smaller chunks of work, known as user stories, and how we refine them just in time before starting the work. We aim to have one Epic in progress while having another Epic ready to be started once the first one is completed. This process is continuous, with “agility” achieved through a just-in-time approach. Consequently, the activities outlined in the workflow are carried out concurrently with the development activities of the current Epic in progress.
The workflow can be sketched as follows:
Introduction and Q&A Session
Slicing the Epic
Breaking Down The Epic
– Stage 1: Review the Slices
– Stage 2: Create User StoriesJust-in-time refinement of User Stories
Introduction and Q&A Session
Once the next Epic has been pulled into our Analysis column, we’re inviting the team to an “Intro & Q&A”. The team is encouraged to look at the item before this session, so they can already think about potential questions. When we have the session, the Product Owner explains the goal of the Epic.
This session isn’t intended for the Product Owner to share their screen and read the Epic’s description, as our team members are fully capable of reading it on their own. Instead, we engage in a discussion about the purpose and problem behind the Epic. It’s a brief 30-to-60-minute call to ensure alignment. We expect everyone to come prepared, using our time efficiently.
We may not have answers to all questions, some may require further investigation. The goal is to be adequately prepared to start the breakdown session.
Start the Thinking Process
The Q&A session is also intended to get the thinking process started. We cannot come up with brilliant ideas on the spot. However, after the Q&A we should have a decent idea about the problem, and people will start thinking (consciously or unconsciously) about how to approach it. Often a lot of ideas pop up in between the Q&A and breakdown sessions.
Slicing the Epic
After the introduction session, the Product Owner spends some quality time dividing the Epic into slices. The slices are intended as incremental reviewable steps, that work as checkpoints towards achieving the end goal of the Epic.
An example for a slice is the “default workflow” (or happy path) that we want to focus on first, to see that it works. The next slices could then be about additional workflows (things like “export & import”) or how the functionality looks like for different users.
Slicing up the Epic serves several purposes:
First, slices guide what the Product Owner envisions as meaningful incremental steps towards the achievement of the goal of an Epic, without focusing on the technical aspects of how to solve the problem at hand.
Second, slices guide the Developers while breaking down the Epic: instead of breaking down the whole Epic in user stories, which might feel daunting and overwhelming, the Developers can focus on breaking down each slice. In our experience, this is an enabling constraint, allowing the team to be creative to solve the problem in the best way.
Finally, slicing is not intended as a one-way street. The slices prepared by the Product Owner are a proposal that is discussed with the team at the very beginning of the next step — the user story mapping session. Whereas usually the slices are not modified, proposals are welcome in the quest for the most effective way to incrementally achieve the end goal proposed by the Epic.
#NoSlices?
We experimented without the slices, and the results were not satisfactory. The team struggled to break down the work. We assume that without the constraint of the slices, what could be done was too open. Having the slices gives guidance while leaving space for the creativity of the team. In any case, we encourage you to try out different approaches.
Breaking Down the Epic
As the team is more familiar with the goal, the next step (usually a few days after the Introduction and Q&A session) is the user story mapping session, where the Epic is ultimately broken down into small units of value (for us those are items of type “User Story”). Always keep in mind that, whereas it is lovely to have an Epic rigorously broken down in user stories, the goal is not to achieve 100% coverage or completeness, but to have an actionable idea on how to solve it and get started. Our plan may not be entirely accurate initially, but we will refine and adjust it as we progress with the work: we aim to get started and have an overview of the work rather than strive for perfection.
Review the Slices
The session kicks off with the Product Owner reviewing the slices. Considering that the overall goal of the Epic has been already discussed, this stage does not usually take too long. The team will usually ask for clarifications, whether a given piece of functionality is intended to be available in this or that slice. Proposals are welcome, and occasionally the slices are updated to accommodate a different approach.
Create User Stories
After aligning the slices, the breakdown stage of the user story mapping begins. This stage might take a few iterations and span across a few meetings. The available slices are positioned in swim lanes and, one by one, each slice is broken down by the team in user stories. During this stage, the team discusses and adds stickies representing individual user stories, while the Product Owner is silent but available in the call to address potential questions regarding requirements/functionality.
Once the team has finalized the creation of sticky notes representing user stories, there’s a moment for wrapping up. Together, we walk through all the identified items to review for any gaps, and uncertainties, and to identify dependencies among stories (e.g., which ones must be completed before others). Following this, there’s a ‘last call’ to break down items into even smaller user stories. While there will be other opportunities to further split the items, it’s valuable to do so while everyone is still fresh on the topic.
The session concludes with the selection of two candidates: one software engineer and one QA specialist. Together, they will create the user stories in our backlog. During the creation process, dependencies to other stories are noted down. This simple yet effective practice helps maximize awareness, reducing the chances of selecting a story for development that may become blocked due to unfinished work.
Please remember that, at this stage, the user stories have not yet been refined from a purely technical perspective. We will need to hold off on that for a moment as we want to discuss something else first.
Is this really User Story Mapping?
If you are familiar with User Story Mapping, you might think that the above description does not really match. You’d be correct, we’ve adjusted the approach so that it makes sense within our context, but still refer to it as “User Story Mapping”. In any case, we recommend everyone to read Jeff Pattons book on User Story Mapping.
User Story Mapping
User Story Mapping is a dead simple idea. Talk about the user's journey through your product by building a simple model…jpattonassociates.com
In Between the Sessions
Depending on the Epic, there is also work happening in between the sessions. For example the team might come up with UI Sketches that serve as input to the next session.
Example Miro Board
The breakdown work is active working sessions. We’re using Miro for this purpose, as it allows us to be flexible and creative. Up until we are ok with the stories we want in our backlog, we don’t need to be constrained by Backlog Management Systems.
Below you see an example of a Board where we discussed a specific Epic. It not only contains the slices and stories but also several ideas about how the UI could look like.
As mentioned, work also happens in between certain sessions. People are encouraged to also add ideas on the board or share some sketches. Having a (virtual) whiteboard available has proven very helpful for our purpose.
Just-in-Time refinement of User Stories
At this point, our backlog contains several user stories that have been prepared jointly by people from the team (the people who do the work). Each user story comprises a descriptive narrative outlining what we aim to achieve within its scope, along with a set of acceptance criteria, ideally presented in a series of Given-When-Then blocks. The involvement of QA specialists at this early stage proves beneficial as it ensures clarity and alignment between the team members.
bliki: Given When Then
a bliki entry for Given When Thenmartinfowler.com
Our development process operates on the principles of Kanban, where we actively monitor and manage flow. As soon as there is space, the opportunity to pull new work into the system, the Product Owner selects the next item to be worked upon — explicit dependencies ensure that user stories are chosen wisely and that risks are taken deliberately.
Such a decision is often taken during our Daily, where we monitor the current Work In Progress and, according to certain parameters, decide if new work needs to be made available*. As with Epics, we do not want to “commit too early” to doing the work on a story. Following a brief final review, the Product Owner pulls the item into the “Waiting for Refinement” column.
*Stay tuned for another post on how we decide if we have capacity for pulling in new work.
Once we start the refinement, for us it’s a commitment to complete this item — the item enters our workflow. Up until this happens, we can still change our minds.
Asynchronously, the Developers pull the item into the “Refinement in Progress” column, where they proceed to refine the user story (or bug, for that matter). Refinement activities are conducted independently by Developers, with the involvement of various team members as needed. If a meeting is needed, the Developers schedule this, but it’s really up to them. During refinement on the story level, we make sure we can answer, among others, the following questions:
Is the goal of this story clear?
Is it clear how we would verify this? Can we verify this (for example do we have the devices needed to do so?)
Do we know how we technically want to solve this?
Are we reasonably sure we can solve this in 10 days or less? (according to our Service Level Expectation)
“Don’t start if we don’t know how to finish”
Optionally, the Developers might break the item down into subtasks to structure the work. While this can be useful, it’s not a must. The tasks for for the Developers, and they can decide if this is helpful for the specific item or not.
The refinement process concludes with a casual review by a fellow Developer to ensure that no significant details have been overlooked and the general approach for solving the problem seems sensible. At this point, development activities on the user story commence as soon as a Developer becomes available.
We want to highlight that work is never assigned to individuals. Work is selected to be worked upon as soon as there is an opening, and it is refined and worked upon by the first person capable of doing so. The Product Owner moves the item till Waiting for Refinement, after that the Developers take over and the Product Owner is only involved as needed.
Does the Full Team Attend these Sessions?
In our team are 7 Developers, and we feel it’s important that the full team joins those sessions. The people that do the work must understand it to do a great job. This is well-invested time for us and we feel the payoff is big. By investing this time, we get better solutions for our problems and more engaged people. While you could do it only with a subset, it’s likely some information is lost.
The output of the breakdown is items in our backlog. But the value is in the conversations we’re having with the team and the fact that the team can shape the work. Don’t underestimate the effect this can have on the engagement.
Learnings
Since we’ve adjusted the way we refine the work, we’ve seen various improvements. Overall, people seem to be more happy. Not very scientific, but it’s fun to work like this, people appreciate being involved and it shows.
Engagement & Growth
In general, the team members are more involved. They actively ask questions, share ideas and concerns, and propose solutions to problems. The fact that they are not treated as “coders” but as “Product Developers” has allowed everyone to grow. They are not order takers, but actively shaping the product, independently whether their area of expertise is Frontend development, automated testing, or anything else.
Programmer- or Product Developer? Why The Difference Matters!
Reflections and findings triggered by the experience interview with Bas Voddemedium.com
Quality
The product we’re working on has quite some history. It’s more than 10 years old, and in a domain that is not very known (electrical power grids). The team inherited a legacy code base, and quality was a bit topic.
It was so challenging, that we invented a forecasting method to include the fact that we’re producing many bugs in the process of creating new features.
Monte Carlo Simulations, Accuracy, and Unplanned Work - A Case Study
In our team, we believe in forecasts via Monte Carlo Simulations over expert estimates. As in our experience, using…prokanban.org
The data suggests that we have been improving in that matter lately. We have a lot fewer bugs that we work on. And the ones we work on are in most cases not related to recent changes, but were there for a longer time.
Self-Management
For our Product Owner, it means less “micromanaging”. He can focus on the higher-level goals, while the team takes care of the “day-to-day” more actively.
And for me as a Scrum Master, life has become very easy. The team has grown a lot, and the process has been shaped in such a way, that in most instances I’m not adding much value anymore. And that’s awesome!
Uncover Better Ways
Please bear in mind that the creation of this process was a process in itself. It did not happen overnight. We tried many different things, some of them failed or proved inefficient after some time.
But all of those experiments were valuable opportunities to inspect and adapt. Please don’t assume that you should do the same nor that it would work as well as it does for us.
We don’t believe in simple “copy/paste” approaches, but we hope we can give you some inspiration on what to try with your teams.
Conclusion
After abandoning Sprint Planning some weeks ago, we left Scrum behind for good. We now focus fully on flow on various levels, be it in the “day-to-day” work or when it comes to the next Epic we want to work on.
We’re very happy with our approach, and as far as we can see, the results are very good. Both on the human side with people being more engaged, as well as on the product side with higher quality.
The foundation of all this is adopting a culture where everyone feels safe to speak up. If you want to continuously improve, you’ll eventually run some experiments that won’t turn out very well.
We can highly recommend starting to experiment with your refinement process. We went from dull meetings that nobody enjoyed to something we’re not only very proud of, but that is energizing the whole team. Often, we get stuck in the “we’ve always done it that way” mentality or think that there are simply no other ways. We beg to differ, and we also think that we as professionals must explore other ways. Will some things not work out? Sure. But it’s a journey, and if you embrace a continuous improvement mentality, you’ll end up with a process that is tailored to your needs. Don’t stop believing.
Shout-Out
I’m happy to say that I played the smallest part in this whole process. I’ve literally learned what our actual process is while writing this post, and I could not be happier.
That would not be possible without a team that not only wants to be more involved but also doesn’t hold back with feedback if they are not happy about something. Kudos to you: Grzegorz, Ewelina, Oleksii, Sumit, Maksym, Nikita & Oleh. Without your willingness to deal with all our “great” ideas and occasionally telling us that it might not be so great after all, we would not improve at all.
Furthermore, a special thanks goes out to Chandan, Michal & Lorenzo for believing in what we’ve been doing and helping to build the culture that made this possible.
You’re making work fun, keep being awesome ❤
