AGILE APPROACH

Agile approach to build software has proved to be very efficient and has given us a huge boost in the last 17 years from its inceptions. The further advents of the agile-specific frameworks like SCRUM or Kanban have formalized the process even more and allowed us to finally have more clear methodology on how we work and engage client.

However, after more than 10 years of practicing SCRUM we have realized that there are some deficiencies to the process, and that we are now in the moment where maybe some improvements of the SCRUM process is needed, or upgrade to a new version maybe?

I have pondered over this earlier and was always aware of different challenges I was facing implementing SCRUM with real teams on real projects, so for that matter I believe it has its merits to discuss this in one place to some extent to help fellow colleagues who, I am certain, are facing the similar challenges all over the planet.

There is also a paper that I have co-authored with prof. T. Zaimovic for ICES 2018, you can find it in this link here: https://tinyurl.com/yeyv9edd.

SCRUM DEFICIENCIES

The SCRUM framework originates from year 2007. I still believe that SCRUM was of a great importance to us and it is surely the way to go when building software, just we should not shy from pinpointing its deficiencies and it is okay to try to seek areas of improvements. This way we can improve this process to suites us better and make us more efficient. Probably the best way is to add some small-scale improvements and then experiment and see how it works in practice, and then to pivot back and change again if it does not prove worthy enough. But what if it does? We should then promote those changes and put it into the mainstream, so to say, but there is no mainstream really. The community is totally decentralized and there is no central body governing the further development of SCRUM. This is also good, because SCRUM being the part of a much wider Agile movement refrains from being too prescriptive, but then we also lose the coordination and hence have to tackle the deficiencies ourselves. And I am sure we are doing that in many companies in the worlds in the same time, trying to solve the problems that are at least 50% the same everywhere.

I assume you have the knowledge of SCRUM and furthermore that you have a working knowledge of it, because it makes no sense to discuss the roles and ceremonies of SCRUM here. I would just like to point out some of the challenges when implementing the SCRUM process and then to try to propose some solutions that I believe might work, based on my own experience.

Duration and Changes

In my experience too many PMs and developers wants to have a spring planned for, say, two weeks and then to be left alone to just work on it. No interruptions, no changes, just doing a planned work. After a while the SCRUM turns into a mini-waterfall method, with all the negative consequences from it.

It is so easy to proclaim embracing the change, but so hard to really do it in a real life. Change is disruptive, you lose focus, you waste time, you need more time to re-focus on something else, and I agree with all of these points. Still, the life is about constant change and we have no way but to accept it as a constant in our lives. The best way is probably to give guidance on how to tackle changes in a most effective way and how to shuffle the task in sprint to gain maximum efficiency, and to also explain to client that changes are always welcome but more changes more waste of time and consequently the money.

Long Term Vision

Running in iterations has the advantage on always focusing on delivering small incremental values to the project, but you lose any longer-term vision. You still have to somehow plan for at least middle range milestones, so I have seen many people and organizations use MS Project to plan for the major milestones and run SCRUM in between those. SCRUM at its core does not support this longer range planning.

DevOps Integration

People claim SCRUM fits DevOps, but I am not so convinced. I know, many would say there is nothing in SCRUM that contradicts to DevOps per se, and they would be true, but in a real-life projects it is really hard to support for DevOps using SCRUM. The reason lies in the very substance of the SCRUM – it is well suited for the work you can plan for, at least to a certain degree, but it fails miserably when facing the work that cannot be planned. Issues such as performance spikes, various challenges in the configuration of environments, monitoring of production and incidents are just a few examples of such.

The spring becomes less important as DevOps brings about daily or hourly deployments, so the issues with not being able to hold ceremonies is exacerbated even further. How can you have reviews, when items are already flown down the pipeline and are already in the production?

SCRUM framework must adopt new approach in order to account for the advent of the DevOps that is becoming prevalent in software development process, and for the reason.

No Saying in Production

I have heard many complains about developers and others in SCRUM teams do not have any saying on what the product is going to look like, and they just have to follow. For me it is impossible to agree with this, and this one stems from not having enough knowledge about the SCRUM process. Product owners are those who identify the functionality of the product that is being built, but I have never heard a client who do not want to have an input from software development team and especially from those who understand the product well and have in depth knowledge of the technology that is used for building that product. It is only a lack of motivation and sub-optimal communication that prevents us from being involved in product design process. This is in no way deficiency in the SCRUM framework itself but comes as a result of poor implementation on our side.

People also tend to forget too often that SCRUM teams should identify the architectural issue that the system they are building has, or some performance and other improvement they want to make and include that in the next spring without consulting the product owner specifically. By doing this we foster the ownership with SCRUM teams and put the onus on them to understand that they build this and own the product as well.

Dealing with Defects

Again, nothing specific about defects in SCRUM, but this is such an important topic that we need to have a definitive insight into this. Unless we assume there should be no defects like ever?

Managing defects is challenging, and it highly depends on where the defect was found. Was it found during the review process? Maybe code review by an architect? After checkin by an automated process that runs as a step in CI? Or further down the pipeline when running automated suite of acceptance or integration tests? Should we create new PBI or re-estimate an existing one? How does that affects the metrics and what if it overflows the capacity of the team? There are many challenges with this and since defects are a fact of life there should be at least some guidelines or best practices spelled out officially or semi-officially on how to tackle those. Right now, I have seen at least two dozen ways people are handling this.

Management Heavy

It is true that SCRUM is really management heavy, and apart from the standard ceremonies we usually end up with a lot of meetings, which still the focus from teams and make them very inefficient.

Definition of Done

I will make a bold statement here when it comes to the definition of done and say this: software is never done and we should stop doing metrics on this. Show me the software application or system that is done? It is like asymptote in math, which defines a line that a curve approaches as it heads towards infinity, likewise the software application or system has the same tendency toward the definition of done, but it never reaches it. Can’t we add more functionality to that application? Or more tests? Or make the most used methods faster and more reliable? More defensive coding? Produce better documentation? The software is never done, we just have to decide when it is good enough for this phase, and project managers should not zero in on this as it can quickly become the point of frustration and kills the creativity with developers.

None of this is handled in any way in the SCRUM framework.

Process Documentation

SCRUM has no string documentation, nor it should. However, the practice has shown that it is too descriptive and about principles, yet people need stricter guidance for most of the activities in it. So I do not believe we should have a strict book of rules about SCRUM, as it would be impossible to envisage all possible situation and even if we would do that it would kill the creativity and fun and nobody would like to work in a system that is so rigidly defined, but a healthy portion of guidance that was filtered from the actual experience of implementing SCRUM in a real life could be very beneficial here.

Mixing Methodologies

Mixing various methodologies for the purpose of breeding a better one should be a familiar concept. We should investigate this more, from the perspective of the SCRUM framework itself, with relation to Lean, Kanban and others.

Embracing Volatility

If SCRUM is used for software development, I think we should be more open to the idea that software development is very unpredictable exercise and that not everything can be accounted for. There will be new issues emerging down the line no matter how much effort you put into “planning everything”. For that reason, product owners and everybody else included in the SCRUM process need to understand that estimates are really guestimates (to some extent), planning goes only so far, unexpected will happen, and it is all just a fact of life. This is not in any way written in the SCRUM process, nor is spelled out being the spirit or culture of SCRUM either. I think it should be.

CONCLUSION

I hope you have gained at least some information and insights on the SCRUM framework from the real-life situation, and that you can maybe see what others are facing and that you are most likely not the only one who wrestles these issues.

I still firmly believe that SCRUM is the framework of choice, but we need to have a focused and much deeper discussion on what does not work so well and what can be improved, and then to do it. The lack of central point of governance is a clear downside here, but then again it is an advantage not to have one governing body because it would inevitably constrain us. The idea that I have in mind is that if those “agile thinker” people would somehow exchange the ideas with people from tranches who are fighting battles every day using SCRUM, it would yield some tangible results that could be accepted by most of SCRUM practitioners and then applied with benefits for all.

Many people would argue that SCRUM gives you broad guidance and you should figure out yourself what exactly to do, but I think broad is just too broad for me. We cannot start improving SCRUM until we realize that there is a significant room for improvement, and maybe that is the main reason why we are not looking at Scrum Next Version for now.

My questions is really this: if SCRUM does not need improvement, then why does even IBM fails 40% of its project (as they do not meet their demand for schedule, budget and quality)? Read more about it here.

There is some progress in this regard, as Scrum Values were added to Scrum in 2016: Courage, Focus, Commitment, Respect and Openness (read about it here)….but we should further optimize and polish this great framework.

BIBLIOGRAPHY

Zaimovic T., Galijasevic M., Efendic A., “Life after Scrum – where next in framework development”, ICES 2018, Sarajevo, October 2018