This document serves as a review and reflects on activities and processes performed during the project in the course “Agile Development Processes” (EDA397). It contains personal opinions, conclusions and feedback on the group's work-flow and puts forward aspects which could most likely find application in later projects. This “Post Mortem” finalizes with a conclusion on the group's communication patterns and reflects on learnings taken from working in an Agile fashion.
Processes & practices
This section will mention and briefly discuss Agile practices and processes used throughout the project some of which have been dropped after a few Sprints.
Scrum methodologies: Roles, Meetings and Retrospectives
As a group we tried to follow all processes defined in the Scrum methodologies which can be divided into pre-game, game and post-game with planning, realization and reflection activities. The most heavily used processes prescribed were Daily Stand-up meetings and Sprint Retrospectives. We used Daily Stand-up meetings twice a week when we met in person and any other days virtual via Skype and TeamSpeak. Meetings were held with all members present as long as no good excuse was stated by a member.
In regard to roles defined, the Scrum Master was selected randomly so that everybody got a chance to occupy the role. The Product Owner was virtually everybody in the group as we tried to enable anybody to incorporate his vision into the final product. Lastly, standard methodologies such as Backlogs, relative estimation and User Stories were also utilized and turned out to be essential for coordinating work.
Critical reflection on processes
The processes having the most recognizable impact on the project were as mentioned Daily Stand-up meetings and Scrum Retrospectives. They both bolstered communication and therefore common agreement on issues and ideas for the product. In addition their value heavily outweighed their time consumption due to their predefined time frame. Just as the combination of Daily Stand-up meetings, User Stories and Backlogs gave a decently clear outline of what every member was working on at any time during the project.
Initially the group's velocity was set to six points but was later changed to about 10-12. Furthermore, the concept of a static velocity defined as an average of previous Sprints was left behind after the second Sprint. As a result, everybody was allowed to add User Stories at will during a running Sprint when his workload allowed him to do so. This after all changed the workload from being pre-fixed by Sprint planning to being more flow-oriented. After all this was perceived positively and did not put up any stress by raising the workload.
Criticism might be put forward as the Backlog sometimes felt being too unstructured to contain all User Stories without any further grouping or dependency feature. Moreover, it does not show the status of a story or task assigned to somebody in the middle of a Sprint. This hitch was however mostly solved by the more informal Daily Stand-up meeting at the start of each day. Furthermore, it would have been useful to dedicate some time for reviewing the remaining Release Backlog after each Sprint during e.g. the Sprint Retrospective.
Accounted by the sometimes rather prescriptive nature of Scrum, a partial application of its methodologies would, from my personal point of view, give a rather phony impression. Still, daily meetings and overall the concept of a Backlog alone already might have a positive impact. Apart from this, User Stories themselves should mostly be changed to become less ambiguous and might also be put into phases visualized in the fashion of a Kanban board. This would help outlining the division of tasks and responsibilities among the team whilst amplifying flow and minimizing overhead created by the need to verbally communicate statuses. The Pivotaltracker somewhat tries to address some of these needs but also failed to easily integrate into each Sprint's work-flow overall. A different UI, a tighter integration into the process of programming and issue tracking might already resolve these pitfalls for the most part.
Lastly, the Definition of Done can be found in our resources repository as well as the Coding Conventions chosen. These governed the group throughout the project even though it was sometimes tempting to discard them partially due to the project's short lived nature. Which should not imply that they are of minor importance for bigger and more long lived projects.
A technique used to a rather lesser extend in our group was Pair Programming as part in the eXterme Programming methodologies. The goal of this process is to fertilize communication and learning while avoiding programmers to get stuck or focus on wrong aspects of a problem at hand.
Critical reflection on process
During the project's start everybody had to make himself familiar with the Objective-C language and the platform surrounding it while one group member already gained lots of experience in developing for the Apple Platform in general. This led to the agreement among all members that initially working and learning separately on simple features while asking everybody for help as soon as problems came up would be more beneficiary. Pair programming was therefore considered as a practice but not heavily used due to its limited applicability in a group with highly varying and more importantly limited knowledge towards the platform. However, this shall not create an impression of solitary group working sessions. Quite the contrary, later tasks/stories were assigned to pairs while everybody else was always open to discussions or giving help. It was just not two members per keyboard with one typing and both interactively solving a problem. The challenging aspect caused by the absence of Pair Programming turned out to be to prevent members not from getting stuck or moving away from common agreements made. Mostly, open discussions, issues and pull requests on our GitHub repository obviated this difficulty.
The most important learning to take away is to embrace help and communication among team members in whatever fashion appropriate. We discovered an adoption of Pair Programming to other forms of coordination and communication to be more efficient in our case.
One more technical process used during the project is Test-driven development. To be more precise the group used Specta and Expecta both proclaiming a Behaviour-driven development (BDD) approach. The group chose BDD over a plain Unit-Testing as a matter of taste and ease of use.
Critical reflection on process
Test-driven Development turned out to be particularly helpful for validating any database interaction which we had to write from scratch due to missing ORM solutions in the Objective-C world. The integration with the IDE (XCode) and our Continuous Integration Server on Travis CI helped us to get notified on failures. It hereby enabled us to allow members to work in somebody else's code as long as it still follows the previously defined specifications. It is important to note, that a test-suite is obviously never complete and will therefore never guarantee completely error-free code. The group was aware of this pitfall and worked in an best-effort manner. Nevertheless, not all parts of the application turned out be testable by code. Mostly UI related issues and bugs were only discovered manually. At the same time they summed up to a quite vast amount of bugs overall which could only be fixed by manual testing.
Interestingly, with BDD, Travis CI and Hubot the group almost drowned in the variety of tools chosen which was why later Hubot and Travis CI were dropped as their significance was minor in contrast to work needed to fully get them to work. Time efficiency for TDD/BDD seems hard to measure as its real advantages shine mostly compared to its absence when code breaks or does not work as expected. This leads to teams having to spend a unpredictable amount of time finding causes and solutions in the dark. On a positive note, it can be motivating to finalize a component with faith when its comprehensive test-suite passes.
In conclusion, it turned out to be extremely valuable for critical and central components such as database interaction. It nevertheless, remains open how to test UI related flaws automatically instead of using manual click-throughs.
Distributed version control (git & GitHub)
This process is not strictly considered to be part of Agile Software Development itself but lobbied the group's work in powerful ways.Especially GitHub introduced a range of routines to the group's work-flow which are not even necessarily part of git or any DVCS system itself. This brief section will describe which of GitHub's practices and features turned out to be particularly useful for and where issues emerged.
Critical reflection on processes
As would seem natural, version control itself enabled us to properly keep track of versions and changes made to code by anybody. Moreover, a branching model with feature and bug branches guaranteed a tidy master branch which contained a build-able version of our product at any point in time.
The nature of git as a DVCS prescribed the approach to commit often to one's local repository while pushing to the remote as soon as a significant validated chunk of development was finished. Furthermore, GitHub advices the use of pull requests to steward unmerged branches (mostly features). This encourages public code reviewing by all members by just commenting or even actively refactoring the code base in a pull request's branch. This turned out to be particularly useful for tracking the status of a branch until it was finally merged into the main branch (the Tag-Browser as an example). Moreover, using GitHub's issue-tracking system integrated nicely into our work-flow where member's discovering a bug would just forward it to the person responsible (e.g. an still open issue). The person would try to fix the bug, close the ticket and the maker will be automatically notified (e.g. a closed issue). Closing tickets is also automated by a specific format for a commit message (e.g. fixes issue #13).
Still, DVCS and GitHub tend to have a flat learning curve for developers being only familiar with centralized version control systems such as SVN. Mostly understanding git's nature and behavior takes some moments and can also be quite frustrating at times. Nevertheless, the benefits will eventually overweight the hard road one has to take. The group commonly agreed on its advantages and quips in previously used CVS systems and as a result does not want move back to using them.
The time spend on tasks and activities varied during the course of this project but were almost homogeneous among all group members. There was a pretty flat learning curve in the beginning of the project which is understandable taking into account that four of five group members had to learn an new language in Objective-C with all its specialties and frameworks as previously unused platform. Especially learning and reading up on the language took some extra effort in the beginning. Nevertheless, the learning curve got steeper as we made ourselves familiar with the platform and group programming helped session helped getting up to speed.
The group met every week for a whole day on Tuesdays and and Thursdays for about 8-10 hours. However, individual work was carried out by every group member from home whenever needed or our schedule allowed one to. Overall, the time spend per week on the project was approximately 20-30 hours. Again, analyzing the commit history and code ownership shows that every group member owns and produced almost the same amount of SLOC. Overall the repository contains over 1000+ commits and total of 9000 SLOC.
Obviously most time was spend on plain coding, followed by feature discussion and code review and refactoring. Interestingly, the least time was spend on Sprint and feature planning whose discussions led9 to quick and agreed decisions among all members. Notwithstanding, the absence of a product vision from the beginning sometimes led to quite different perceptions and understandings of the final product. Which as a result led to corrections and arguments about finished features while already working on new ones. Even though this is an integral part of Agile Software Development an at least rudimentary, cohesive vision from a UI perspective in the beginning of the project would have helped to mitigate some of these issues.
Group communication and interaction
First and foremost, communication and interaction in this group was almost frictionless right from the beginning of the project. Observations mentioned in this sections were small adjustments made geared towards mitigating minor glitches.
First, after simply splitting up tasks to assign them to individuals in the beginning, it was later decided to form subgroups. These were arranged for the most important User Stories in order to prevent members from getting a tunneled vision towards their tasks which in the end do not comply with the common understanding. In addition, after the first Acceptance Test (AT) the team decided that the Scrum Master to be responsible for the presentation during the AT which also included knowledge about all stories completed during the Sprint. This demanded a final rehearsal before each AT in which members presented their completed stories to the Scrum Master. The Scrum Master then puts stories into the "delivered" status and can also decline a story to not be part of the AT.
Moreover, after having disagreements in regard to expectations with the User Interface the group decided to evolve the UI through mock-ups - visible in the resources repository. This reduced later inconsistencies and lowered potential mismatches of expectations towards certain aspects of the application's UI.
Lastly, the group tried to follow the means of asynchronous communications as mentioned later. After dropping Google Groups as a medium for discussions in the beginning, the group moved to HipChat integrating it with Hubot and Travis CI to get notified upon build- or spec-failures. Here, the group-chat alone had tremendous advantages for generally coordinating work.
Reflections on project
This section discusses the project's agile approaches pros and cons in comparison to a theoretical impact of following techniques according a waterfall/plan-driven process.
One of the most interesting aspects of developing a product in an Agile style was the presence of an evolving feature set. At most points in time the group was not genuinely focused on laying out the complete vision of the project right from the beginning. The iterative approach here clearly showed its presence in comparison to a waterfall/planned-approach. The team was in constant feedback with itself and tried to use other members as an imaginary product owner(s). As a result, initially developed parts of the application where changed until everybody was content with the overall result. This was most apparent for the application's UI. Even though maintaining unpredictable changes and the consequent uncertainty was a problem at times it still seemed mostly easier than rectifying flaws caused by misled and false planning. Shortsightedly, open discussions and multiple iterations may appear more time consuming than just blindly declaring functionality. On the long run, the satisfaction of all members after finishing the whole project outweighed this opportunity cost. Moreover, it leads programmers focusing upon one step at a time without being constrained by future, depending steps and their inherent assumptions. To be more exact, we tried to communicate meaningful steps/changes in one's own work to others for acceptance or to discuss possible changes.
In addition, the group tried to develop independent components which integrate easily with one another. By the same token, Objective-C's strictly prescribed MVC-architecture forced standardization among everybody's code and thereby also enhanced its comprehension. Admittedly the integration still became a bigger task over time and needed more and more code reviewing by team members to make components reusable and properly interact with one another.
After all, processes advertised by Agile Development sometimes led to an idiosyncratic form of chaos which needed some supervision at times by the team as a whole. The Backlog and its User Stories were the aspect needing most central governance. Maintaining them to lower their ambiguity was a challenge. Most notably by the fact that most of them were written at an early stage and picked later while revision was only applied to a subset at irregular intervals. In addition, their nature of being a stripped down requirement without any compulsory additional information, seems to be their strength and weakness at the same time.
Learnings for later projects
This project, regardless of its size and experience of its group members, clearly showed benefits in practices courted by Agile Development. Mostly, Agile showed its strengths when working in pairs or alone in an asynchronous fashion. This procedure elevates its benefit when open discussions and critical reflection towards other team members' work took place. Here, communication and interaction between team members might sometimes need to be enforced. Most importantly programmers have to accept others comments and eventually even refactor their work without taking it personally. At the same time, Agile sometimes enforces programmers to leave their comfort zone, adapt new processes and work with new tools whenever necessary. There should be an openness between all team members being ready to help while at the same time postulating collective code ownership and responsibility about everybody's own work. In this fashion, compulsory company meetings are more than often negated with reference to the asynchronous approach of self management. Coordination and a common agreement on an iteratively developed product vision can be achieved with repeating meetings in which anyone can raise concerns and impact the entire product's direction.
Furthermore, Agile distinctively seems to shine for “product software” and may be some what hard to adjust towards the needs of commissional work. This argument is founded by the fact that commissional work is mostly deadline focused and needs clear time estimations. Time estimations which define the cost of a certain feature and thereby enable the customer to integrate the product in their own business while managing their expectations. Agile can possibly be adapted towards solving these issues but one of Agile's foremost best practice is not to estimate time but relative complexity.
Lastly, Agile processes have a well known difficulty to scale towards bigger team sizes and corporations. This problem has been addressed by various parties from different angles. The principal lesson to take away seems to keep teams small, bureaucratic overhead at a minimum and allow teams to choose their tools themselves while demanding a certain degree of homogeneity to preserve interoperability. Moreover, coordination between members should not be accomplished by a strict hierarchy but more by special roles among team members using flat and friction free, osmotic flows of communication.