Posts filed under ‘Business Analysis’
Peer Reviews – Give a Little, Gain a Lot
Oh it’s been a while since I’ve posted and so much has changed. I’ve recently taken on a new role with my existing employer to lead an organization that is focused on the full life-cycle of IT – IT projects and Application Support, plus corporate collaboration/knowledge management just for fun because I love it! It’s been 60 days in my new role, so it’s time to get back into writing….
In a recent team meeting, I introduced my new team of project managers, business analysts and quality assurance analysts to the practice of peer reviews. Their first question was “what is it” and then their fears of exposure came to light. Fair emotions as who likes to be judged? Like many teams, this team puts their heart into what they do. So yeah, who is going to enjoy the idea of being in front of their peers and judged for something they put their hearts into? Not me. As this was not my first time introducing this topic to a team, I was prepared. I believe it’s good to let the teams get this initial reaction out of their systems – then I move them towards the ultimate goal of peer reviews – learning.
Peer reviews are a great way to extend knowledge across your team members without the formality of training. Formal training is a must-have on many topics but there may be functions that you feel your team could come to alignment on without the expense of time or money related to training. I am also not a believer that every process should conform to a specific standard as sometimes creativity used to build deliverables can be a strength to the team. But how do you share these creative, superior deliverables with others on the team so they might learn or be inspired to try something similar on one of their future endeavors? And if you do have a situation where you do need 100% conformity to a standard, peer reviews can be a safer way to apply these policies than having an audit process (especially one done by a manager) defined.
Establishing a strong peer review process can be done with 3 guiding principles:
- No judging – by anyone
- Everyone plays
- Measure what counts
No Judging – By Anyone
Many groups in IT think about peer reviews and think of code reviews. This is a chance to make sure standards are applied and good design practices utilized. While this is definitely an option, you can also consider making peer reviews more about learning. My recommendation, if applicable for a team you are growing, you might want to consider asking the team to use the model for the presenter to gain insight from their peers about other areas of the business they might be more knowledgeable about than the presenter. The presenter can also look at the opportunity as a way to grow the business acumen of their peers and to share a technique they used that their peers may not have used before on a project. But in this model, there is no judging – no tracking the number of mistakes made by the presenter. No notes taken on the fixes made – unless the presenter wants to record them. An additional concern often raised is that the presenter feels exposed if their manager is in the room. Easy to solve – don’t be there. I have never sat in a peer review meeting for one of my teams. If not having me present eases the pressure on the presenter and perhaps reduces the desire of some of the audience to show off their knowledge, then it’s an easy thing for me to do for everyone. In reality, if I am managing this team, I have access to the deliverables and I can review them anytime. What is more important to me when growing a team is that the team feels supported by each other so they turn to each other in both the peer reviews but also through unstructured methods.
Everyone Plays
Participating in the peer review process should not become a burden for your team. I rarely make this a requirement before the finalization of a deliverable. Learning should be easy – not another hurdle for your team. But if learning is important, you need to know that it’s happening often enough for your team to be growing (both as a team and in their knowledge). So consider the frequency that peer reviews should occur, and how often you want to make it required for your team members. And depending on the size of your teams, it may not be feasible for everyone to participate in all reviews. This is a decision you have to make depending on the make-up and maturity of your team. I typically opt for optional engagement BUT each person has to participate in at least 1 peer review process per month, quarter, etc.
Measure What Counts
In my model, peer reviews are about building a sense of team, knowledge of methods that can be used in deliverables, and a start to alignment in some processes. If this is what counts, then this is what should be measured. Sense of team can be measured through how participation is occurring. I do ask teams to track who is participating in each meeting so I can look for trends. If your team members embrace this model, you’ll see that they are excited to join the meetings, and not avoiding them. You will have outliers but those might be addressed case-by-case to find out why they are not joining the discussion. Knowledge of methods can be explored in 1:1s by asking your team members as they produce deliverables if they are applying anything they have learned in a peer review session. This is a great way to introduce approach planning as well if you do not discuss this with your team members already (don’t just start a deliverable but what model/methods will you use as a first step). Finally, alignment which can be harder to measure. This may be something you have to analyze by reviewing several deliverables. You also have to wait for a period of time to let the peer review model work. You may also have to seek feedback from your team in regards to areas that are very out of alignment that they would like to work on. Once they have seen various approaches, they may be the first ones to speak up and mention that they feel a certain amount of alignment would benefit their audience. How great would this be versus having management require alignment? While I am not shy about requiring certain things, if I can let my team come to these concepts on their own without putting our audience/business at risk, this is method has a much higher and faster adoption rate than management telling them to do it. And in this case, since I am managing a set of PM/BA/QA team members, doing a role that I have done in the past, it may be seen as me forcing my way onto them, which is never popular. This way, they are finding their own way – as a team.
In Summary
Peer reviews are nothing new though I have not seen them exercised across all areas of a business nor even all areas of IT. They are a great tool for building stronger teams, awareness to new methods, and better deliverables. If you take a very intentional approach to introducing this topic with your team, you are bound to get great results.
Does your team use any form of peer reviews? Are they known for being judgmental or do they facilitate positive experiences in learning and team building? Share your stories as I would love to hear about your experiences so we can all weave them into our next opportunities to introduce this topic!
Test Execution Management – More than Pass/Fail
Your project is in the home run stretch. You have a few weeks before the deployment and testing is underway. Do you know what the odds are of deploying on time? One strong indicator is your test execution statistics. You are collecting them, right?
Testing is more than test cases and marking them as pass or fail. You need to have attributes about your test cases in order to have strong test execution management. Without this information, when crunch time hits and you need to revise your test plan to hit your date, you will be shooting from the gut and not working off facts. That can work out – but do you want to bet the bank on it? Or your career?
While building your test cases, here are a few things to consider.
1. Which requirements does this test case link to? And are they “Must have” or “Nice to have” This information will help you set priority on your test cases.
2. Negative testing – good – but at what probability? If you know that a situation “might” occur and you want to test for it – that’s a negative test case. Every project should have them. But if you know that the probability is low of this situation occurring, you can leave this test case for last instead of using valuable resources to get it done first.
3. Relationships – it’s not always about who you know…. How a test case relates to another is important. For instance, if you have a defect on test case #2 and it relates to test case #3 – shouldn’t you re-test both of them when the defect is resolved? YES – but without this mapping, will you neglect #3 and not realize that it’s now not working as expected.
4. Execution schedule – this was always my least favorite but necessary item. If you have 3 weeks to test and in week 1, you have only 15 out of 200 test cases completed – are you in good shape? What will you tell your management team? Do resources need to work the weekend or extra hours? Like any project, having milestones along the way will help. A goal to have x% of test cases done by the end of week 1 is a good start. Publish statistics to your team and ask them to help provide ways to keep the effort on schedule. Publish the statistics to your project manager to help as an input in determining the status of their project.
Testing – it’s not the glamorous part of a project. It’s the part at the end that has to be done and is usually challenged with a shortened timeline as the other phases ran over. Do not let your project status be subjective – capturing a few details along the way can make the difference in saying “Yes we can hit our date” or “I hope so….”.
What other attributes do you capture to help with test execution management?
When to Derail from your Software Development Life Cycle
I’ll admit it – I like the Waterfall software development life cycle (SDLC). Really – who wouldn’t? Everything all defined up front, designed, coded, and delivered in a tidy package – its the perfect scenario. But it’s not realistic. Waterfall can take a longer timeframe. How often can the business define what they want in month 1 and not have their needs change by month 8? And is a single delivery window the best option for your business? However, I have not heard of the perfect scenario accomplished using Iterative or Agile either – nor any other SDLC.
In reality, what makes a great project when it starts with a SDLC and is evaluated along the way by a Project Manager and team. The ability to evaluate the situation and determine the best points in the SDLC to take risk by introducing change and when to stay on the straight and narrow.
As I type this, I am chuckling as there are people that know me who are probably dropping their jaws about now. They know that I do like to drive a team towards staying on the defined SDLC path. If the risk of the defined path is low, the reward for change is low, and the results are being made with the defined path, I struggle with why to take “perceived” short-cuts. The downside to taking an alternative approach with your SDLC is that everyone has to understand the revised plan and potential risks to scope, schedule, and cost.
In the end, this evaluation is not unlike any evaluation a business has to make when considering a change. How disruptive could the change be to our people? Do we have the skills needed to support the change? What is our potential reward? What will it cost short-term? What will it cost long-term? Will we have re-work to do if this goes wrong? How soon can we evaluate if the change is effective? And many more….
Many IT organizations diligently follow their SDLC and I am sure they have found great ways to deliver quality projects. I think it is unrealistic to assume that every project and team can do this without having Project Managers ready to evaluate the scenario and work with the project team to adjust the plan as needed to improve the odds of success. As IT leaders, we should evaluate our Project Managers’ ability to do this evaluation and talk to them about how to evaluate when its appropriate to consider a shift, how to evaluate the risk & rewards
Don’t Skip the Review
It’s done. A picture-perfect set of requirements are documented. Blood, sweat, and tears have poured into this document – most of them yours but maybe some from your stakeholders along the way. The next step is to host a review. Yes, you are going to make everyone involved in building this document (and a few others – keep reading), walk through the documentation one more time.
Yes…. I am serious. I know the review process may seem like overkill but there are so many reasons to do it. Before you forget – we don’t live in a black & white world so I do accept the fact that it may not be appropriate for every project so use your judgement but keep some of these points in mind.
One Last Chance
Your stakeholders have provided you with many requirements and insights throughout the analysis phase of your project. But do you have any guarantee that you have them all captured – or even captured accurately? A walk-through of documentation gives everyone a chance to talk about any questions or concerns with the documentation as a group. This may trigger their thoughts on an additional requirement. A requirement identified in an earlier phase of the project will save you costly time and money if identified later – like in testing or worse, in production. So yes, it can be costly to pull all stakeholders into a review meeting, but it is more costly to identify scope changes later.
Missed a Meeting – Never Happens
Did every stakeholder participate in every requirements gathering meeting? No? Consider me shocked! Work (and life) happens and you cannot expect everyone to be in every session. Plus, with other elicitation techniques such as one-on-ones, job shadowing, etc, some people may be left out. Having a documentation review lets you breakdown silos across teams and gives everyone a chance to hear the requirements gathered.
Now to Make it Productive
Once you believe there is value in hosting the meeting – here are some tips to consider before setting it up.
- Do not schedule a marathon review – if your documentation is lengthy, consider breaking up your review in 90 minute or less periods over a short period of time. Maybe a session in the morning, a break, and then another session in the afternoon. Perhaps you need to spread it over days – host a session each day. Putting too much time between review sessions will reduce the value of the meeting as people will lose continuity of the discussion and their ability to focus.
- Publish the documentation at least 2 business days in advance – ask for your participants to come prepared with feedback. Make it clear to them that you will not be reading the document to them word-for-word – so they need to have read the document, and prepared notes for questions or concerns.
- Offer flexibility in logistics – face to face is great but not always realistic – offer participants the chance to dial into the meeting if located remotely but keep them engaged in the conversation – not just on mute while they read their email.
- Multiple Presenters – I’m sure you have a lovely voice but ask others involved in the project to help host the review. If there is a section that a stakeholder provided input on, ask them to present that portion of the document. It will get others actively involved and breaks up the meeting a bit for everyone.
- Notes – it is very hard to host a meeting and take detailed notes. And since this meeting is all about updates to the documentation, notes are important. Ask a co-worker or stakeholder to help out by taking the notes during the meeting. If you can swing it, make updates to the documentation as you review the documents. That reduces your post-meeting work AND lets your stakeholders see your precise updates.
One last point that you might be thinking – yes, you could send out your documentation and ask everyone to read it and provide feedback to you. Does everyone always do exactly as you ask in the timeframe that you ask it? If so, I am very jealous! People get busy – and they don’t get to reading documents related to projects not set to deploy for another 30-120 days. You also lose out on the rich, interactive conversation that could arise if a stakeholder points out an issue with the documentation. I also recently saw there some IT team members brought in some complimentary documentation to the review to show the stakeholders – this was a great use of time, in my opinion and helped the stakeholders see/hear that 1) IT was listening, and 2) IT is already moving forward to meet their needs. Could not happen if the review had not occurred!
Just my perspective but my experience has shown that hosting a requirements documentation (regardless of requirements format – visual, contextual, use cases, etc) review. What experiences have you had with reviews? What worked well? What did not? Share with everyone so we never have to sit through a boring review again!
Using Color on a Project Status – Worthless without Definition
Red, Yellow and Green – infamous colors used for street lights, ugly holiday sweaters and project management status reports. While I am fundamentally opposed to the ugly sweaters, I am a strong supporter of street lights and project management status reports. The status report is a critical communication tool.
Project team members and leadership within your company cannot check on project status daily. Yet they need to stay informed of the overall health of the project. There are so many ways to do this in a way that is effective for team members, executives, and the project manager. This post is going to focus on the use of color in status reporting but another day I may have to write a post about how to use the status report (PLEASE do not have a meeting to read your status report….. ). I would strongly suggest looking into the PMI organization for some good guidance from their PMBOK (Project Management Body of Knowledge).
I understand why so many people are driven to report status based on color. It’s easy to read, visually easy to recognize, and so many people like to use jazzy little clip art of spotlights on their status report for fun. But a red, yellow, green status report is as worthless as the crayons it is made with if you do not define what specifically drives a project to be classified by one of these three colors.
All projects involve 3 things – Scope, Schedule, and Cost. Therefore, a good way to define how you will monitor the status of your project involves these three areas. You can try to blend these three areas together to report a single “color” status — but I think the overall project is better reported by area for a more complete perspective.
Reporting Status of Scope
Scope is a bit tricky as it is either defined, or not. A good way to monitor the health of your Scope is to track the number of pending, approved, or rejected change requests (post-requirements approval). Given that a change in Scope will also (likely) impact your Schedule and/or Cost – this area bleeds into the other two. I would suggest talking with your Project Sponsor early in the project to define what an acceptable amount of change will be for this project. This can be hard to quantify but when you consider what the priority of the project is (Scope, Schedule or Cost – only one is most important) – then you should be able to set an objective to manage scope change appropriately. If Scope is your project priority – then you may expect a high number of change requests. If Schedule is your priority, you may want to define an objective to only accept X days of additional development into your project through change requests. If Cost is your priority, then you may state that change requests will be budgeted to stay within X% of your total budget. Again – this is subjective – just some ideas here to get the conversation started.
So for Scope, in our situation, we may state that the following:
At any point, if the volume of approved change requests is within 0-2% of total budget = the status of the effort will be Green. Change Requests +2%-6% will be considered Yellow. Any point when the approved change requests exceed 6%, the status of Scope will be Red.
Fact-based and yet still associated to a color! Best of both worlds!
Reporting Status of Schedule
This type of status reporting is more native to many project managers. It’s all about dates. Most project managers define a WBS early in their project lifecycle. The idea of reporting status is about using the major milestones in your schedule and noting when you are starting to press against the boundaries that you define early in the lifecycle.
Any major milestone that is not met within 2 business days of the Planned Date will cause for the Schedule to be reported as Yellow. If a major milestone is not met within 5 business days of the Planned Date will cause for the Schedule to be reported as Red. Any major milestone that is accomplished prior to or by the Planned Date will cause for the Schedule to be reported as Green.
If, along the way, the project schedule changes (which they so often do) – then just revise your tracking plans as needed. And make sure you define your own boundaries – I used 2 & 5 business days as an example – it all depends on the Project Priority. If Schedule is not a significant objective for this project, pick your battles and select windows of time that are appropriate. Don’t scream that your project is Red because of a missed Major Milestone when, in reality, the Project Sponsor just wants it delivered sometime this year.
Reporting Status of Cost
By now, I’m almost thinking that I don’t need to write anything for this section. I’m sure that you get the idea….
The status of Cost will be reported as Green if Project Actuals & Project Forecast are within 2% of the Total Budget. If the Project Actuals + Project Forecast are between +2%-6%, the status of Cost will be reported as Yellow. If Project Actuals & Project Forecast exceed 6%, the status of Cost will be reported as Red.
In Summary
Hopefully this post is giving you some ideas of why colors are meaningless unless you put some thought behind them. Too often, project managers apply color subjectively and can get everyone’s attention on a RED project that may be distracting management attention from something else that needs it more. If you are going to use color, define how it will be applied in advance. Make it part of your project kick-off so everyone understands how the status will be reported and the color will complement their understanding of the situation being reported on the status.
