Agile experience in complex projects

    Recently I've taken small break to rearrange my knowledge and experience gained in quite difficult projects that I have last 2 years. The mainstream of this period is continuous attempts to apply agile approachs in our team. Upon a time my endeavour ceases to look endless: We really indeed can be agile! (it's not obligatory so popular SCRUM).
    Recent XP camp in Kyiv (Thanks so much to Naresh Jain) and works of Gojko Adzic (I wish I knew that man earlier) made me strong believe that even our heavy project can be more "agilized".
    Presentation "Agile experience in complex projects" is an excerption of ideas I embrace about software development and particularly about project management. Of course, it was technically infeasible to look so broad topic through all aspects, but my purpose was to give an overview of subject, since I met so many IT people that never experienced problems like that, especially if they never participated big teams.

Q: How did God manage to create the world in 6 days?
A: Well ... there were no previous versions to maintain backward compatibility with.

    My presentation's focus is so called complex project. I've introduced this term myself, meaning following:

Distributed enterprise application
Distributed team in locations and cultures
No solid corporate culture
No men, who know entire system

    Few people among our community have already investigated problems and got the conclusions pretty close to mine.
    Although I've caught chill 2 days before, I hold presentation confident and quite successful. Questions of attendies discovered even more topics for future research.
    I heard question requesting patterns and antipatterns for agile-oriented project management. Although I like antipatterns very much (It was my first presentation at my host company) I didn't intend to discover "patterns" in this presentation. But the question is still very leading indeed: probably next post I'll devote to discovered antipatterns for project manager in my model. I do have a plan.

Self-managed teams

    Recently, project manager in our team has been changed. My new boss is quite diligent in applying strict software development process. It's wanted and needed indeed: it's a really good thing when you know that some certain functionality will be delivered before certain date. At least management can sleep well in this case. But what chance of the dreams to become true in the team that steadily delivers many unexpected things, both good and bad. In complex projects work of developers are pretty much research work: everything brings piece of chaos to the complex system.     So all costs should be put under consideration not only in aspects of men-hours, but also in some very human-based and subjective opinion about increasing entropy. To be simple: each time you add new functionality you should think about chaos you make to your application. I will probably talk more about this effect later. So if we know that men-hours are not scalable (special thanks to Frederick P. Brooks), entropy increment is something even unmeasurable. In environment when nobody knows system, each development task means investigation, design, coding, testing, testing and testing ...
    Ideal developer for this is good motivated specialist that can "keep himself busy": he commits whole cycle from task specification to integration and he does not disdain to use any technologies due to solve any tasks. Although division of labour market still has people like that:

    I don't know exact reason for that. Probably IT are still very attractive not for just welfare, but also for self development.
    Such developer can be not so keen in each of his things he does, but he is much better for developing some kind of software than team from several people engaged in separated things:

• he consumes less management resource than team of specialists of many sorts. It is really cool - the management resource is is inclement deficit.


• he tends to self development and quickly embraces knowledge of applied system


• he and people like that provides good base to create self-managed teams

    If you have more than 40% percent of people like that in your team, team becomes more or less self-managed: they able to distribute tasks themself, coach people who are not skilled enough. Broad specialists can also maintain whole development cycle protecting software from ambitions of others.
    There are also some disadvantages:

• There are might be lack of people that likes formal management (with burndown charts, timelogin, issue tracking, road mapping and so on)


• They like too engage themselfs with tasks they want. Remember that common developer designs only that things that he can and wants to create


• They start fighting windmills when face challenging problems

    This means that team still need force that leads the development. This not always can be done by customer or head of developer. Customer and head masters can be so distant from development team that team doesn't feel leadership from that side. As it was recently discovered manage IT people is similar to herding cats (Last year I'd got this amazing book).
    Usually good developer asks somebody if he needs extra information about what to do and also he asks if he needs to make decision how to do. Many developers that are still with strong technical background doesn't like others to asks. So on one hand we need two roles:


    1. Somebody who knows what to do and prevent to do things developers supposed to not do.

    2. Somebody who knows how to do and prevent to do things in incorrect way.


    So on the following scheme I establish two more roles - architect and analyst:

    Analyst is pretty close to the meaning of analyst in all other models of development cycle. But destinctive features of architect that he is not just IT specialist that creates architecture or (even worse) designs software, but man who can reply on question how to make software. This not obligotary means that he should no everything, in fact this would be insecure for enterprise, and doesn't mean that he should create development document accordingly to standards himself, but he gains role of the knowledge base moderator. His artifacts is every document that helps to define how development team suppose to resolve their tasks, including guidelines, solutions, designs of crucial units in system, description of used technologies, that are not necessarily created by him, but in the good environment carefully selected by him from sketches created by team, external sources and self experience.
    Architect doesn't response for design anymore: it should be responsibility of people that creates software (the team). But he should be very responsive on questions that helps team to make decision. And the most important thing is that architecture not more process that bound to specific time or stage in project: architecture as development document should be steadily updated as new experience is incomming to the people that make software. This is very distinctive, because this means that we are not able to say "Oh, men! Congratulation - you have successfully passed the architecture stage, let's come for planning!". Because there is no more stage of architecture, it's continuous activity.
    What impact has aforementioned approach on analyst?
    Quite similar to one that has on architect: in complex project we can expect surprises of inconsistency of requirements or infeasibility of them analyst at any time within period of development. So architect is one who not only creates documents, but also use them to explain others what things are expected from software. Like architecture, requirement analysis is not more determined stage, but more or less continuous process. Analyst comparatively with architect should have probably longer gap between his start and start of the development itself, but boundaries between pure requirement analysis and support requirements are anyway obfuscated ...

Macros in Visual Studio

I was stumbled by some trouble in Visual Studio 2008 today:
arghhh! macros refused to run!
Even the simplest code in VB demonstrates that VS doesn't run the code (Even if you put break-point on the single line):

Sub HelloWorld()
DTE.ActiveDocument.Selection.Text = "HelloWorld!"
End Sub

Only thing I did recently was that I installed several Windows Updates on my machine. And the same happends to my Visual Studio 2005.
Only post in MS Support knowledge base was not up-to-date. At least I had different symptomatics with this. In an hour cursing policy of continuous updates I run Visual Studio 2008 installation and forced it to repair.

Indeed that helped for VS 2008, but not for VS 2005

Challenging Visual Studio

After three month of silence I returned to my blogspot with fresh ideas. The most essential reason for this is that I was preparing special courses for our internal team and was busy with this as well as with managing our development process.

Recently I've discovered .Net framework 3.5 and Visual Studion 2008 for me. Thank God and Microsoft that irrespective to dynamic UI changes in popular Microsoft products development environment is kept conservative: It will cut down at least efforts put on getting used with new IDE.

So, what we have here:

• WPF (Windows presentation foundation) - something that can make development of UI in web and on the desktop faster and more flexible


• WCF (Windows communtication foundation) which can be used for transmitting data from one location to another


• New language features (LINQ, anonymous classes, expression trees and lambda-functions


• many features for ASP.NET


• Office and Sharepoint integration features

To some negative clues I can put following:
Reduction of Composite UI and Microsoft Enterprise Libraries. Sounds like vendor set 2nd priority for these technologies.

In next posts I'll put more attentions at the new technlogies, but right now my general conclusion after overviewing new development environment is following:
Microsoft tends to move accent to Web-development, but maintaining strong position in desktop application by providing easy passages to integrate .Net applications to the Office, Sharepoint and other business solutions

Much better than one

    I have noticed one tool that I use rather often.
    I have laptop used both at home and at the office. Both at work and at the office I have desktop computer. Whenever I run some automated process on one machine I intensively work with another. It increases efficiency: Instead of waiting until NAnt and MSBuild will do their job and let me go ahead, I able to read, write, send/receive e-mails, prepare presentation and even develop on other machine. That's can be really big advantage for you, if you're already disappointed to have strong beliefs in multitasking OS.
    Until I installed synergy, it was very annoying to transfer some small amount of data between my computers (for instance, URL or connection string from PC to laptop). Even if you have instance messenger on both or well organized set of shared folders. My hands where getting tired because I should work with two keyboards and two mice.
    So, let's start with small practical introduction: So you have you're laptop (in my case PC-BORYS) to the left side of your desktop PC (BORYSDESK) with network connection established between them. Synergy is installed on both PC and laptop.

    It's convenient to share PC's keyboard and mouse, so consider laptop PC-BORYS as client and BORYSDESK as server. Here is main screen of synergy:

    On client all you need is to specify server name (either network name or IP), see figure above. On server you should set adjacent screens configuration. In my case it's two screens connected side-by-side like on figure below:


    Press start to share keyboard, mouse and clipboard on two PC's!

    You can't expect synergy to transfer windows and support drag-and-drop operation between PC's, but there are anyway quite useful features ...

Advantages:

- synergy can work with any amount of PC's (imagine if you have 2 laptops and one PC)
- clipboard buffer is also shared. You can copy-paste from one PC to another
- configuration is saved
- synergy doesn't require administrator rights on PC and perfectly works under normal user
- clipboard buffer can be transferred with any kind of data (pictures, doc files and so on)
- tray icon with connection status is provided

Disadvantages:

- when server is very busy, mouse and keyboard can freeze
- It incorrectly works with different keyboard layouts and functional key (so it's good when you use one variance of latin alphabet)
- It gets stuck if you place large amount of data in clipboard buffer

NB: It's open source, freeware project, which hasn't been changed since 2006. The latest version is 1.3.1

dotProject experience

    dotProject is light weight, free web-based team collaboration tool.
Hosting where I have tried it first was linux-based with cPanel X v2.3.0 as primary remote control. I'm not very good in administrating Linux server, but Fantastico tool installed dotProject in a few clicks. Of cause, I wasn't able to install specific version (I had experimented with , but as for me, it is extremely important that installation can be so easy. Anyway, team collaboration system is something customers don't pay for. And if you're collaboration tool don't demand your efforts, it's big advantage both for development and management.
    It's free, web-based and written in PHP. There are some entities you need to know about if you work with dotProject:

company - your team
projects - your global tasks, that make sense for management
tasks - your local tasks, that do not make sense for management
files - heterogeneous project documentation
forum - place where we can discuss something
tickets - some notes, that can either trigger discussion or task
events - calendar events
user - project contributor

dotProject has module structure. Each module can be represented as tool bar item (or hidden for convenient or security reason).


    Basic installation include following modules: Companies, Projects, Tasks, Calendar, Files, Contacts, Forums, Tickets, User Administration, System Administration, Departments, Help.
    For project like ours some of them are in fact surplus, so first I did is getting rid of Companies (we operate just one), Departments and Help. I was about to remove projects also, but realized that no task or event can be created without activating this panel. That's really strange, because logically I expected that like file can be created in files panel, or user in User Administration, tasks should be created on Tasks panel. This is little disadvantage: you can't set user rights to let them create tasks without removing or adding projects. Another inconsistency I have noticed is that user should have rights to operate forums to have right at least to post their message. Administrator can set although rights to create entities, and restrict to remove, but I think in most cases it's not a good solution.

dotProject in use

How to create user users in dotProject?

    Each user has rich profile with contacts (even ICQ is available), but which can not be extended with custom fields. To create user you should use User Administration panel and you can actually fill whole necessary information about them
Users should have role to be active. I reduced rights for project worker (by default project worker has all rights except administrative panel (User Admininstration and System Administration). dotProject proposes few standard user roles for different project contributors.

    User can be united in departments, project and companies, but I found this functionality somewhat surplus for me.
    There is a confusing field in user's profile called user type: it has no impact on rights or settings, but used only for information.

Tasks and tickets
    dotProject is not much convenient for doing this, but indeed that's good that tasks are more "heavy" than ticket, so tasks should be created from panel (there are 3-4 clicks needed. There are some attributes in tasks and projects that are not necessary: project can have states, tasks can be different types, and set of those states and types can be only changed by intruding to DB. Both tasks and project have budget, it's good that dotProject has this feature, but this field can be hidden for different roles, which makes this feature not very good for practical usage. In general all entity attributes in dotProject is not objects of administration, so you can't grant access to some entity with concealing some data for them.
    Tasks can be distributed by multiple people, but this requires from manager to see percentage of contribution, so probably it is reasonable to agree with team how to use percentage, otherwise it will be a mess.

Tickets supposed to be used as trouble reports, but can be used for "offline brainstorming" - people can express their proposals here.

Forums - are not very special discussion placeholder. Each forum has moderator, which has the same rights as anybody else, so this is not role, but just administrator

Files - that's really cool that dotProject has storage for homogeneous documents (in any format). It supports versions (and check in/check out operations) and allows to view the documents directly from browser.

Any version can be reviewed and all versions are kept in file system on the server. With mySql database which can be exported from server, it makes dotProject quite portable. File can be attached to certain task and project or not attached to any.

Calendar - good for organizing events. Events can be created by users in project pane and notification will be sent if needed. Actually after fanstastico installation notification works! Like most of entities events can have also web URL (for collaboration with Wiki) and bound to certain task

dotProject tuning

Other settings - there are plenty of settings, in System Administration panel even color scheme can be set, but there are anyway many properties that can be easily changed: for instance although you can set Saturday and Sunday as working day in calendar, you can't make Sunday in the end of week. Settings for notification and mail server can be set here,
custom fields can be attached to companies, projects, tasks and events, but unfortunately not to tickets and contacts.

It took 3 hours to install dotProject domestically on my Windows machine, but I'm lucky I did it: I have deployed our team DB on my machine and made my experiments even more thoroughly. DB include some significant settings that can be useful for dotProject customization (for example task and document categories and user types). They are represented as rows in certain columns of mySql database.

Other features

• Contacts can be imported and synchronized with LDAP


• Translation manager can be used for localization


• User logs allow tracking user activity in project (Good feature for paranoiac managers)


• There are more modules like search, history, install and so on available in System Administration

Let's briefly resume problems in dotProject

  There are some trouble with Cyrillic characters, which can make impediment for export/import DB or incorrectly rendered in PDF reports.
  I have found SQL injection error (discovered when I included to company name apostrophe character, see figure to the left)
  Calendar can't be set to European standard
  Initial color scheme is ugly, IMHO
  Fast localization and some attributes choice require changes in DB
  dotProject provides no integration with CVS systems, probably connection with development workflow should be conducted by filling some custom fields to tasks or making direct reference

Main conclusion: dotProject provides really good start for small and medium IT projects with good team collaboration platform which can be extended or in far future transferred to something and can be easily used together with Wiki or other CMS system.

Can team spirit be automated?

    Two weeks ago the remarkable workshop was conducted at our company: about different tools that may be used in different aspects of the IT projects. I was presenting CruiseControl.Net as build server and continuous integration tool. Other report was about Notepad++ - actually very good editor to amend configuration files. The third report was an overview of team collaboration tools given by our the most experienced consultant.
    His task was really challenging: web is flooded with different solution regarding team collaboration and managing IT-projects. It's really difficult to come up with something the top most convenient on this. IMHO, this tendency took unstoppable power from banal wish of every two from three developers to become a project manager. Another reason is that there is no good solution for managing IT-project indeed. Not all projects that have the modern infrastructure for collaborating have success. Among this, I have seen couple of successful projects were neither team collaboration nor bug tracking nor even source control systems were used. Accordingly to our gurus Tom DeMarco and Timothy Lister the most valuable thing is team spirit, which can not be replaced by tools or methodologies. Their book about peopleware is issued in 1987 and against all odds still lives after many collaboration tools are dead ...
    Day by day, moment by moment I am being persuaded there is no way to replace team spirit by some tool, like poor architecture can not be replaced by superb UML designing.
    Well let's turn back to our workshop: then only one tool attract my attention - dotProject - it was said to be free, light-weight and full functioning. With Trac that also is very remarkable (our company once had it as default tool for IT-startups), it is going to be something I'm gonna start future investigations.
    I felt I badly needed some place where I can see myself evolution of the team and evolution of the team collaboration.
    Two weeks after I suddenly get an opportunity to set some experiment with both Trac and dotProject. The first in line was dotProject, so in next post I am going to share experience about this ...