Understanding Requirements through Filtering, Negotiating and Shifting

Abstract: Traditional requirement engineering approaches pay little attention to how the requirements are interpreted and shared by different parties in an organization. Our study suggests that requirement shaping during a project can be described as a process where attitudes and expectations are filtered, shifted and negotiated repeatedly. We studied a large e-commerce platform development project and observed that preconceptions, attitudes and expectations about systems development among project participants filtered the understanding of software requirements, negotiating between project participants resolved the issues caused by filtering and shifts in these attitudes and expectations facilitated changes in the understanding of requirements. The study provides a new interpretation of how technology is collectively interpreted in organizations.

Keywords: Requirements engineering, requirements understanding, socio-cognitive processes

Categories: D.2.1, K.6.1

1 Introduction

Studies on large software projects show that they fail at an unacceptably high rate. Even if a software project is completed in time, it produces software which is of heterogeneous quality and often exceeds its budget. Many of the problems encountered during systems development are attributable to shortcomings in the software product's requirements. The problems reported in earlier research on requirement engineering typically involve the following issues: insufficient user involvement, ambiguous, changing and incomplete requirements.

Intensive research on software tools, modelling methods and processes for performing requirement elicitation has not yet delivered tools or techniques that could guarantee foolproof success in software projects. Traditional requirement engineering (RE) approaches offer poor understanding of how to specify and manage requirements for large evolving systems, how the requirements are understood and what kind of problems exist in the commercial practice.

Recent research directed into the social and organizational processes in requirement elicitation sees it as an emergent political process [Bergman, 2002] constructed through conflicting interests and agendas, resource constraints and political influences or as a socio-cognitive problem solving process [Orlikowski, 1994; Davidson, 2002]. This socio-cognitive problem solving process is characterized by differences and repeated shifts in assumptions and expectations of technology, which disrupt the project participants' understanding of requirements.

2 Research subject

We studied, using a grounded theory approach a large-scale e-commerce platform project with internal customers within a large telecom operator. In the beginning of the study we observed that the requirements did not stabilize during the project but instead kept changing, causing problems and delays for the project. This led us to study how the software requirements were shaped and interpreted during the project.

The study was carried out in a software development department of an international ICT company. The development of new applications and services was assigned to an in-house software development unit (Internal Development Unit, later referred as IDU) or outsourced companies. The use of IDU for development of new services was mandated by the company's top management. IDU had approximately 150 employees that had formerly focused on R&D work in the company. During the past few years it had tried to improve its software skills and processes in order to make its development more effective and also to prove its capability to other business units. All the business units of the company did not agree with IDU's processes and did not trust in its software development capability. Their attitudes towards IDU competencies in software development were quite suspicious, mainly because of IDU's history as an R&D department. Quite often business units preferred outsourcing instead of developing in-house.

The business owner of our case study project was the business unit (later referred to as BU) responsible for marketing new business ideas related to electronic and mobile commerce services. The project developed a mobile commerce service platform. The system was intended to enable organizers or their sponsors to promote their products in all kinds of events, such as ice hockey and football games. The system was composed of two subsystems: the platform in which the services were running (Platform subsystem) and the toolbox, which allowed adding, configuring and simulating these services (Tool subsystem). This toolbox was intended to run in a Windows PC and the service platform in UNIX environment.

3 Findings

The project highlighted problems in current approaches to requirement elicitation and systems development in general, which still largely assume that projects proceed with distinct phases and more or less in a waterfall fashion from a vague understanding of the idea of the system into a concrete system. Instead, we observed that the requirement understanding was filtered by project participants in the beginning of the project, negotiating between project participants resolved the issues caused by filtering and shifts in these attitudes and expectations facilitated changes in the understanding of requirements.

This process of filtering, negotiating and shifting can be described as an ad-hoc and iterative process where the software requirements unfold during social interaction, communication and negotiation between parties.

During the analysis, we observed four categories of attitudes and expectations that affected the understanding of requirements of various project participants. The identified categories were:

• Business value of system development, i.e. the attitudes and expectations about the relationship between business and system development.
• System development strategy, i.e. the attitudes and expectations about the suitable system development life cycle model and processes.
• System development capability, i.e. the assumptions and expectations about competencies in different areas of system development, such as user interfaces and databases.
• System development resource allocation, i.e. the assumptions and expectations about scheduling, budgeting, and priorities of systems development projects in time-to-market pressures.

Within these four categories, we identified a process of stereotypical “tensions” that had important effects on how the project participants understood requirements in different phases of the project. We named these tensions as:

• Filtering that occurred when a stakeholder of the development process left something out of the scope because of his/her understanding, attitudes, expectations, or experiences
• Negotiating that tried to resolve the incongruence between stakeholders. Incongruence happened when understanding, attitudes, or expectations differed among the stakeholders, causing conflicts and misunderstanding. After negotiating the understanding of attitudes and expectations were the same
• Shifting that took place when the understanding of a frame changed. After a frame shift, the parties involved got an understanding of a frame that was more aligned with and suitable for the current situation than before the shift

In the beginning of our Electronic Commerce platform project differences in attitudes and expectations redirected the participants' attention away from the relevant information and filtered their understanding of the project requirements. In the later phases, the ability of the project to resolve these differences in the negotiations between participants redirected their focus towards the relevant information and led to shifting in their attitudes and expectations helping the project to make sense of the information in a new way. This sense making in the system context was an iterative ad hoc-process that happened through social interaction, communication and negotiation between the parties.

The results of our study contribute to existing requirement research in an important way. This study makes a substantive contribution to the understanding of the requirement elicitation process and systems development in general. While the current approaches still largely assume that projects proceed with distinct phases in a more or less waterfall fashion and the system is developed from an understanding of the idea into a final system, which satisfies the originally stated requirements. Instead, our study implies that the requirement shaping is an ad-hoc and iterative process in which filtering, negotiating and shifting of different attitudes and expectations about systems development change the participants' interpretation and understanding of requirements during the project.

The full description of the case and the results can be found from [Ovaska, 2005].

References

[Bergman, 2002] Bergman, M., King, J.L., and Lyytinen, K. "Large Scale Requirements Analysis Revisited: The need for Understanding the Political Ecology of Requirements Engineering", Requirements Engineering (7:3), 2002, pp.152-171.

[Davidson, 2002] Davidson, E. J. "Technology Frames and Framing: A Socio-Cognitive Investigationof Requirement Determination", MIS Quarterly, (26:4), 2002, pp. 329-357.

[Orlikowski, 1994] Orlikowski, W. J., and Gash, D. C. "Technological Frames: Making Sense of Information Technology in Organizations", ACM Transactions on Information Systems, (12:2), 1994, pp. 174-207.

[Ovaska, 2005] Ovaska, P., M. Rossi and K. Smolander. "Filtering, Negotiating and Shifting in the Understanding of Information System Requirements", Scandinavian Journal of Information Systems (17:1), 2005, pp. 31-66.