Politie, ICT en Wetenschap

Niemand lijkt verrast over het rapport aangaande de invoering van BVH binnen de politie. Het is een projectfalen zoals deelnemers aan het project vermoedelijk al van mijlen zagen aankomen, waarvan er al zoveel zijn geweest en met dezelfde oorzaken die in tal van artikelen zijn besproken. Hoe onbevredigend.
De discussie over het rapport binnen de politiek laat zich raden en ik maak me geen illusie over de rol die focus op verbetering daarbij zal spelen. De wijsvinger zal het voornaamste onderzoeksgereedschap zijn. En toch, een betere case om inzicht in de complexiteit van grote ICT projecten te krijgen is moeilijk voorstelbaar. Ik zou graag zien dat de wetenschap een actieve rol op zich neemt om de dynamiek die bij dit project is ontstaan te begrijpen en de vervolgstappen die gezet gaan worden te onderzoeken.
Ik ben heel benieuwd welke tak van wetenschap zich geroepen voelt om hier verantwoordelijkheid in te nemen. Dit projectfalen is immers een multi-disciplinair falen. Het is het falen van de politiek, van de bestuurskunde, van informatica. En, minstens zo belangrijk, zijn de economische, sociologische en psychologische factoren.
Een unieke kans voor de wetenschap om zijn waarde voor de praktijk aan te tonen.

Semat: On theory and mathematics

There is quite some discussion on the applicability of mathematics to a theory of software engineering (SE). I think that at the core of the discussion is confusion about the role of theory and mathematics in understanding a phenomenon. I want to share my idea of that role.

The purpose of mathematics is to model real-world behavior. The mathematical model is based on theoretical insight into the relevant variables governing the behavior. The benefit of the mathematical model is that it can be used for quantitative prediction of the real-world behavior. Therewith allowing for verification or falsification of the theory. Falsification can mean that the mathematical relation between the variables is incorrect or that the variables themselves are incorrect. Scientific progress goes in the direction of increased theoretical understanding of the relevant variables and the mathematical relation between those variables.

It has been stated that Semat wants to define a theory based on mathematics. Given my understanding of theory and mathematics I would rephrase that to: Semat wants to increase theoretical understanding of SE expressed as and supported by a mathematical model of the dynamical behavior of SE systems.

I suggest the following steps to reach this goal (which btw I highly support):

1. Define the real-world system that is the subject to scientific investigation, theoretical explanation and mathematical modelling. That is: Define what we mean with SE.
2. Identify an initial set of variables that are candidates to describe the dynamical behavior of the system. This identification should be based on current understanding (theory) of SE.
3. Use the identified set of variables to describe the dynamical behavior of  many, diverse SE instances. In the process refine the set of descriptive variables.
4. Identify variables that explain the differences in dynamical behavior between the SE instances. Manipulation of those variables should result in different dynamical behavior.
5. Define a mathematical model that describes the relation between the explanatory and the descriptive variables.
6. Continuously refine the mathematical model using the scientific method. My expectation is that, like in weather-forecasting, model and theory will increasingly overlap.

The value of a mathematical model resulting from this approach will be tremendous. To name just two that directly relate to the frustration Semat is based on: Such a model will allow to predict the course of SE projects and it will make it possible to distinguish between the quality of methods and practices.

Because of the breadth and complexity of the subject this is a long road. And it is therefore not at all a surprise to me that the general feeling is that little progress has been made to date. In my opinion Semat will be a success if it triggers the scientific journey expressed above starting with the definition of the system under investigation.

Semat: In need of enthusiasm, support and momentum

Semat has expressed the ambition to refound Software Engineering (http://www.semat.org/pub/Main/WebHome/SEMAT-vision.pdf). The tentative definition of Software Engineering (SE) accepted during the Zurich meeting (http://www.semat.org/pub/Main/SematZurichMarch2010/Zurich_meeting_report.pdf) is:

The discipline of making software systems deliver the required value to all stakeholders.

As an interested observer, after all I also feel that SE can do much better, many things can be said about what comes out of Semat thusfar. At this point of time only one question is of real interest: Has Semat started off in a direction that might result in refounding SE in a way that allows to make software systems that are able to deliver more value to more stakeholders?

The problem that I see now is that what comes out of Semat sofar simply does not make my heart tick any faster. I’ll try to analyze why I think that is. And will then indicate what I would like to see to believe in the success of Semat.

Initiating the project to create an SE product

In my earlier post I suggested to start-up a project to release a Software Engineering (SE) product. A product that supports SE teams in being successful with any kind of SE project. I will apply the project approach that I consider fundamental and that I have outlined in another earlier post. The main steps of that approach are:

1. Assess project complexity
2. Define project risks
3. Define project phasing
4. Select initial project members
5. Share all information with the members
6. Allow members to self-organize into a team
7. Manage project context

In this post I will make some initial brainstorming remarks on what I consider of importance for the first three steps.

Semat? Let’s make software…

There is an interesting initiative going on to redefine the foundation of Software Engineering: Semat. Semat is a collection of the brightest software engineers and software engineering methodologists. Like with any collection of bright people there somewhat of a struggle to get the train rolling, to agree on an approach that allows the experts to collaborate and excel. I want to contribute by offering my ideas on how to make this unique effort a success. Today part 1.

The participants to Semat are typically software engineering professionals and not scientists. They are especially good at finding solutions for problems. Not necessarily at scientific research into the causes for the problems and the reasons why the solutions work. Actually the non-scientific nature of software engineering is what caused this crowd to gather.

Therefore my suggestion is to define a twofold goal for Semat:

1. Define fundamental software engineering lessons learned from decades of experience. Which are the very basic elements of software engineering everyone agrees on? This will be an expression of current software engineering theory.
2. Define topics that require scientific research and invite science to address them.

I propose an approach to accomplish both goals that uses the strength of the specific group of Semat people. I propose to define a project for the Semat people of which the goal is to create software, a product. A product that supports software engineering project teams in being successful with any kind of software engineering project. This approach will provide a context in which the experts can excel (and if they can’t one should wonder whether they are indeed the experts Semat needs) and will trigger far superior collaboration than any conference, workgroup based approach.

If the Semat people succeed in this project, the product will demonstrate the state-of-the-art understanding of software engineering in the form of an explicit architecture, a materialized theory. Design errors in the product will be topics for scientific investigation. New versions of the product will be an expression of scientific progress or of increased application of scientific insights.

Please join me in promoting this approach or let me know why you disagree.

On the Semat initiative

I am very curious about the outcome of the Semat gathering that is currently going on in Zurich. My curiosity has a diverse background:

1. As a software development manager who has always critically reflected on his ability to increase the chance of success of a software project, I’d like to see Software Engineering (SE) mature to a level where it actually gives solid ground for all of us doing software projects.
2. I am afraid that semat will choose to divide SE in many subcategories and that teams will be assigned to investigate those subcategories. A typical, but fruitless reductionist approach when trying to understand complex issues.
3. As a strong believer that complexity theory and non-linear dynamics are best able to describe behavioral patterns such as collaborative software engineering in a business environment I am looking for signs that such concepts are picked up. Proper description of SE behavioral patterns is a critical first step for any scientific investigation and explanation.
4. As a facilitator I am intrigued to see what results the problem statement, the collection of individuals and the approach taken will deliver. On the short-term: Will the first meeting provide a basis to continue at all? On the long-term: Will a process have been initiated that results in a solution to the original problem?
5. As a scientist I am interested in the scientific method applied to come to the scientifically supported basis for software engineering.

Besides these points I am just fascinated by group behavior. I have many questions about all the forces working on this group of people. To name a few: Most people are experts in their domain, some are considered gurus. Are these people capable of forming a goal-oriented group? Will a leader stand up? What leadership approach should be taken to allow the experts to contribute their much-needed expertise? What impact does the extreme openness of the group have on the resulting group dynamics (Just consider the amount of tweets being sent by participants). Is there a way to use that openness to strengthen the initiative? Will the group be patient enough to let the initiative develop, self-organize into a valuable endeavour?

I hope to know more soon…

On software project management

I have more than 10 years experience in software development management. I am rather up-to-date on current literature, blogs, white papers on software engineering, software project management and software methodologies. And I now finally feel that an approach to software project management is shaping between my ears and in my heart. In this blog I will try to translate that to paper for the potential benefit of others.

To start with I’ll summarize the steps. In the remainder of the blog I will zoom in on each step:

1. Assess project complexity
2. Define project risks
3. Define project phasing
4. Select initial project members
5. Share all information with the members
6. Allow members to self-organize into a team
7. Manage project context