The Creation of the PSM II Assessment

In the spring of 2013 I abandoned my position as Principal Consultant at a large international consulting firm to engage in a partnership with Ken Schwaber and his organization Scrum.org, operating under the title “Director of the Professional Scrum series”. Next to and after co-developing the Agility Path framework, from which Evidence-based Management (‘EBMgt’) was derived, and the Nexus framework for Scaled Professional Scrum (‘SPS’), I vividly remember creating the first version of (what became) the PSM II assessment of Scrum.org. In my mind it was in the spring of 2015, but checking out my archives tells me it must have been the spring of 2016. That is most likely an indication of how flawed my memory actually is…

By that time, we observed how many people took the PSM I assessment (‘Professional Scrum Master level I’) and how few people were taking the assessment that was by then known as PSM II (‘Professional Scrum Master level II’). I remember (thereby potentially disregarding how flawed my memories can be) that at that time around 40k individuals held the PSM I credential while no more than a few 100 had achieved the PSM II certification. It felt like we were failing to help people understand Scrum better and demonstrate that knowledge. In practice, mostly people aspiring to become a Professional Scrum Trainer (‘PST’) were actually doing the PSM II assessment (as a mandatory requirement). I remember taking the assessment myself with that ambition in mind. Luckily I wrote a blog note on 8 September 2010 to ‘remember’ (hence: compensate for my flawed memory) that I was #27 worldwide to achieve it.

So, I started creating a completely new set of questions for what we envisioned to become a “Professional Scrum Practitioner” assessment (PSP), a way for people to demonstrate their practical insights in Scrum. The initial idea was to create this assessment next to the existing Professional Scrum Master assessments. It took me several iterations to find direction. Ultimately, I created an assessment with largely two types of questions: (a) questions in the style of PSM I (still rather straightforward, yet more sophisticated) and (b) highly contextual questions. The latter category of questions consisted of a rather elaborate description of a context and practical situation with equally elaborate answering options. This obviously required a lot more reading. The assessment did not contain open or essay questions like PSM II, questions that require assessment takers to formulate their own answers. It was in that sense a ‘multiple choice’ assessment, like PSM I is.

As a last step in the creation process, I had printed out every question like index cards with the answering options on the back side. Colleague staff members had helped me to get the language clear and unambiguous. On one of my visits to the Boston HQ of Scrum.org, I took Ken and Chris Schwaber through them, question by question, answering option by answering option.

If I remember well, we went live as “PSP” assessment although it was soon after decided to rename it to “PSM II” and rename the existing PSM II to “PSM III”, which was more than obvious in the end. I might be ignoring how unreliable my memory is, but I am really very sure that the initial passing scores were horribly low. Only between 40–45% of the people taking the assessment actually achieved the certification score of 85%. Looking into the missed questions turned out that a disproportionate amount of them had to do with a poor understanding of (the importance of) Done and the definition of Done. We already knew that this was also a problem in the other assessments, which is why I had given the topic more focus in my writings. [ Note to self (1): Consider better the long-term effects of making statements like “If Scrum was to be reduced to one purpose, and one purpose only, that is the creation of a Done Increment in a Sprint” to emphasize how “Done is a crucial part of Scrum, actually”. Some people will overlook that you don’t say that Scrum actually is to be reduced to one purpose only.] We kept those questions in and I decided to make it the focus of my speaking engagements. We did remove a few questions regarding the self-organizing capabilities of people and teams, because even within the PST community by then it seemed that only Ken and I agreed about the correct answers. Gradually the passing scores started increasing, albeit still slightly (as could be expected).

I left my position at the company in the spring of 2016. I think the passing scores since then kept increasing, even remarkably. I can’t say why because I haven’t been involved with or consulted by the company since going fully independent in 2016. I only had a strange experience with the PSM II assessment when applying to teach the PSM II class (now called PSM-A) in 2022. [ Note to self (2): this might be a subject for another autobiographical blog note: “How I envisioned creating a PSM II class already in the fall of 2015 but…”] Despite having created the assessment and holding the credential, I remember being told that I had to retake the PSM II assessment as it seemed that I had passed the earlier, “more difficult” version.

Since then, I have continued my work of collecting many of my actual experiences and bundling them into my Scrum Caretaker Book of Exercises. I use those cases-pure, mixed or merged- to help people improve their understanding of Scrum, as additional cases in my Professional Scrum Classes and as the single source of work for my Scrum Pocket Classes (named after and based upon my book “ Scrum — A Pocket Guide “). As I did with the original PSM II questions: I draw from my actual experience and the people I work with and compile them into cases fit for an assessment, often combining different situations I have been in. It sometimes does involve some simplification or shortening of timelines.

Someday I plan to use them to create and launch a “Scrum Pocket Test”.

Allow me to share an example of a contextual question (and the answering options) from my Scrum Caretaker Book of Exercises:

Where is my release?

Situation

A team is developing a highly anticipated new version of an application. The release of this version is planned to be announced in the next quarterly newsletter of the company. This newsletter will be published after the current Sprint (5). The complete functionality of the new version was estimated at 200 points in total.

A few days into Sprint 5, having attended the Sprint Review of Sprint 4, the CIO and the head of product management call the Scrum Master to their office. They share their disappointment. They have no idea anymore when the release will finally be available since:

• For Sprints 1, 2 and 3 it was reported that respectively 7, 19 and 25 points were delivered.
• The CIO already doubled the team in size after Sprint 2.
• Sprint 5 is now underway, after only 28 points were delivered in Sprint 4.

They remind the Scrum Master that a velocity of 40 points/Sprint was promised for the release to be announced in time for the newsletter. They want to abandon Scrum, or minimally stop all development until there is a proper plan.

What is to be done? What would you want the Scrum Master to say? How about the promised release?

(Take some time to think this through for yourself before checking out the options below)

Options

From following scenarios, which one would you want the Scrum Master to follow?

A. The Scrum Master replies that progress and scope are not a Scrum Master accountability. The stakeholders need to ask such questions to the Product Owner, the only one who is entitled to make predictions about the future. The Scrum Master reminds them to certainly join the next Sprint Review again (of Sprint 5), which is less than 2 weeks away anyhow, for the opportunity to ask the Product Owner about the release date in this case.

B. The Scrum Master clarifies that Scrum is founded on empirical process control. Empiricism in a complex environment holds that only the past is certain, and the future is unknown. No statements about future release dates are possible. However, each Sprint Review does offer a particular decision opportunity to release.

C. The Scrum Master accepts that the stakeholders feel lost and that transparency over the actual progress needs to be restored. The Scrum Master says to take this up with the Product Owner and to help the Product Owner explore tactics to give insights into the actual progress to the stakeholders and what might be ready by when (scope vs time decisions).

D. The Scrum Master demonstrates how the proven past should be used to look at the future. A calculation shows that the new release date will be in 4 or 5 Sprints, i.e. the result of dividing the open amount of work (121 points) by the proven velocity (28). The Scrum Master adds that velocity typically increases over time, so that the actual release date will likely even be sooner, in 3 Sprints.

E. The Scrum Master says to understand the importance of releasing in a timely fashion and apologizes to the stakeholders for not achieving the expected velocity and endangering the newsletter announcement. The Scrum Master will inform the CIO of what types of Developers are needed most to expand the team again to increase speed. The Scrum Master assures that this financial investment and commitment by the management is likely to increase the team’s commitment.

Originally published at https://guntherverheyen.com on October 8, 2024.