- Background
- Approach
- Analysis of Data
- Conclusions and Further Work
- Acknowledgements
Like this article? We recommend
Like this article? We recommend
Like this article? We recommend
Analysis of Data
As a precursor to the data analysis, we need a breakdown of the types of companies that were the subjects of the research. In the initial research, 37% were small companies, 32% were medium-size companies, 26% were large nationals and multinationals, and 5% were government departments. The smaller companies had an average of 2 to 4 permanent staff members, with up to 10 part-time or contractual positions. The medium offices had up to 25 employees, and the large offices had up to 145 staff members in Australia (and typically thousands worldwide, with a substantial percentage of those staff members working in research and development).
By the time this article was written, the composition of these companies had changed due to the dot.com crash of 2000. Some of the companies were no longer operating, and many of the medium-size companies had been taken over by multinationals. Overall, 28% of the companies initially interviewed had changed in composition in some form. Thirty-nine percent are now small companies, a further 51% are large nationals or multinationals, 5% are medium-size companies, and another 5% are government agencies.
The initial information that all the companies address, in both the interviews and the documentation, is their primary business and their domain targets. This is usually found at the beginning of the description of the services offered. This is in line with any business purpose in which a prospective client can surmise quite early the suitability of a company. From here, though, company information that is made available varies. There is no correlation between the size of a company and the amount or type of information supplied. For example, only 22% of companies did not present either a portfolio or case studies of their work, yet this 22% is made up of both small organizations and multinationals.
The company documentation that is presented is mainly supported by examples of the company's work. This is achieved through the presentation of either case studies in print brochures or on company Web sites, or portfolios on the company Web site, with live links or both case studies and portfolios. Seventy-eight of the companies listed case studies or portfolios to support their development skills and to give prospective clients examples of the quality of their work and the league of their clients. This is another mechanism for marketing a company's ability and demonstrating the areas of development discussed throughout the available documentation.
Of the respondent companies, 78% described the complexity of the applications they develop, the technologies they handle, and the scale of their projects in their company documentation. Interestingly, the 22% that didn't describe the complexities of their applications seemed to focus more on the complexity of the business goals, expectations, and deliverables of the Web site. Despite describing the complexity of the applications, 12% of the respondents actually dismissed their importance to the clients and encouraged clients to focus on the business outcomes and leave the systems work to the Web developers. This could be indicative of the low level of understanding of Web systems that clients have when they approach Web-development companies.
Most companies (78%) identified the components that clients require, and 56% of companies described their company's project-management approach—whether liaisons are team-based or individual contacts. This seemed to bring the most contradictions between the interview responses and the rhetoric in the documentation. Twenty-two percent of respondents claimed to have a team approach in the interview, yet the rhetoric highlighted a central contact—or, vice versa, with the rhetoric highlighting a team approach and the interview indicating that only one person is the client liaison. This indicates that there is a discrepancy in companies' project-management approaches; one could even go as far as to say that this misrepresentation of company processes is detrimental to client/developer relationship expectations.
The areas avoided in the literature were client levels of understanding. There seemed to be little or no mention of the need for clients to better understand the development process for systems or the marketing or business development of a system. Through their publicly available documentation, only 12% of companies prompted the user to consider areas of development that must be covered in an initial brief or scoping document in some depth. Some examples of these prompts are business goals, risk management, business re-engineering, and marketing requirements. These companies seemed to be preparing prospective clients in the areas of discussion and information required for the initial briefs and meetings.
Issues of changes and evolving requirements were covered by only 28% of the documentation. These changes were a major issue for the companies that were interviewed, as were the processes for managing evolving changes. Yet these issues were hardly touched upon. This seems to reflect more on the purpose of the company's publicly available documentation as a tool for attracting clients than as a tool for discussing in-house processes and issues that could arise during the development of Web projects. Furthermore, evolvability and scalability of projects (as opposed to evolvability and scalability of requirements) seemed to be considered by only half of the interviewed companies. Of the half that did identify this need, however, two thirds identified it in both documentation and interviews.
One of the more significant results is that, in both the available documentation and interviews, 71% of the companies identified the importance of Web site development relating to current and developing business goals and mission statements. Twelve percent identified the need in only the interview, 5% identified it in only the documentation, and 12% didn't identify the need for the business goals at all. This shows that the majority of the companies involved in the research recognized the close association between Web sites and the need to reflect and communicate the whole business mission statement. This is an aspect that traditional software-engineering practices did not necessarily have to consider because their output is not available to users outside the work environment.
Using a methodology was also a high consideration. Some examples of the methodologies identified are Object-Oriented Design, Rational Unified Process, Rapid Application Development, and Joint Application Development. Some of the companies developed their own methodology tools that they then developed into standard pro-forma. Of all the interviewees, 29% made no mention of having any methodologies in place. Forty-seven mentioned having methodologies in place in both their documentation and interviews, another 12% mentioned having methodologies in place in only the documentation, and a further 12% mentioned having methodologies in place in only the interviews.
The issue is mainly with the 12% that mentioned having a methodology in place in their publicly available documentation, yet in their interviews made no mention of using methodologies to develop their work. Stating to prospective clients that the company uses a methodology, but then not actually having anything in place jeopardizes development of highly functional and successful Web sites, as well as relations with the clients. On the other hand, not mentioning the use of a methodology in the interviews might have been an oversight of the interviewee. In a candid conversation with one of the interviewees, the person felt that listing the vague processes of the methodology, while giving away no intellectual property, attracted clients who had a reasonable understanding of systems development and wanted to know that the company they were either hiring used certain processes in the workplace.
The most surprising result was the low level of importance placed on usability testing and user development. Sixty-four percent of interviewees made no mention of these issues. Twelve percent addressed the issue in their publicly available documentation only and not in the interview, and 23% mentioned it in both the documentation and the interview. Only 35% mentioned the need to identify different stakeholders affecting the client outcomes. Of these 35%, two thirds mentioned the need to identify stakeholders in the interview only. Only 5% of the total companies identified stakeholder needs in both the interview and the documentation. And of all the company documentation that was examined for this article, only 5% mentioned human-computer interaction (HCI). Though 25% of the respondents discussed customer relationship management (CRM) in their documentation, again, there was no mention of CRM in the interviews. This could be a reflection of using marketing catchwords: The same 25% of respondents that mentioned customer relationship management in their publicly available documentation also mentioned stakeholders or user needs in their interviews.
General Observations
The primary observation that can be made involves the differing purposes of the material analyzed. The interviews were conducted specifically to obtain from companies their methodologies and specification processes when dealing with clients and specific problems pertaining to scope creep, evolvability of projects, changes in awareness levels of the possibilities of projects, and requirements needs. The publicly available documentation is used as the marketing and publicity information for a company's services. This information is available for the primary purpose of attracting and retaining clients. Some of the companies offer a range of information, from their "methodologies" to the various services they offer and client lists, case studies, and portfolios. Some companies allow their "work" to be their publicity machine and offer very little information about their processes.
Surprisingly, a lot of the information that was collected through the interviews in the area of processes and methodologies used and requirements identified from the clients also appeared in the publicly available documentation. As mentioned earlier, one of the companies felt that giving information on the general processes undertaken while not giving away any intellectual property is a fine marketing tool that attracts the level of clients that the company wants to work with. Accordingly, the companies that outline their processes are presenting their prospective clients with steps that will be further explained at an initial meeting. This certainly supports the statement made earlier that clients often see the existence of a process as indicative of an improved development capability, whether the company actually follows that process or not. Consequently, in most of the analyzed data for the interviewed companies, the processes outlined in the marketing documentation tended to be more comprehensive and more structured than the processes outlined in the interviews.
The implications of the similarity between the interviews and the publicly available documentation are that developers' processes are still in conceptually early stages and are continually evolving. This leads developers to view their processes as components of their marketing tools. Highlighting in-house processes in place puts clients at ease. This is reflected by the 59% of the respondents who claim to have methodologies in place. Of concern are the 12% of developers whose responses in the interviews did not reflect this information, though there is awareness of the client expectation for these processes to exist. There seems to be a notion among the developers that clients need to be placated with assurances that these processes are in place; whether it is through deliberate application or not must still be ascertained.
There also seemed to be two approaches in the rhetoric used in the publicly available documentation. One approach could be called patronizing, in that clients' "technical" fears are allayed and the developers let the client know that they need not concern themselves in gaining the technical knowledge required as long as business outcomes are met. The other approach seems to actively want to educate clients on the extent of the development that will be carried out, the business ramifications of these developments, and the level of awareness that clients will need to enter a signed agreement.
Interestingly, there is no distinct pattern for the types of companies that take either approach. That is, there is an equal distribution of small and national/multinational developers that "patronize" a client as there are developers that want to educate their prospective clients. This seems to reflect a disparate view among Web developers of client approaches to Web development. Both these approaches highlight an industry still in its infancy, trying to work out the level of involvement from either systems and IT divisions or marketing and strategic business divisions. Web development has seen the emergence of a division that involves a hybrid of systems/IT, publishing, and marketing skill sets in the project team required to achieve a successful development.
The areas in which inconsistencies occur among the interviews and documentation can be attributed to 28% of the companies having changed in composition, whether having gone through a merger or a takeover or having closed down completely. Even so, inconsistencies were evident in the remaining 72% of companies that were still functioning in their original form. The most concerning inconsistency is the project-management approach taken by companies. To write in the publicly available documentation that a team approach is taken, but to state in the interviews that the company works on a one-to-one basis or with one point of contact with the client shows diametrically opposite approaches.
Significantly, the importance of usability and user needs that are characteristic of the literature on Internet-enabled systems is not present in either the documentation or the interviews. Though the terms "user interfaces," "look and feel," and "user engagement" are used throughout marketing documentation, little in the documented processes supports their use in companies' specification and requirements processes. Again, this highlights the fact that claimed processes exist in only a superficial form; they are utilized as marketing tools but are not necessarily being embodied as an integral process in the development. Despite the increased importance of usability and quality attributes as applications become more externally visible, this is addressed only briefly in the marketing documentation and barely rates a mention in the interviews.