A Working Project

a bird's-eye view

In order to tie together the lectures in a meaningful way, I'm including a high-level walk through of nearly everything involved in a large scale project for the Internet. Included are examples of the many kinds and forms of information needed to complete a project. Whole books have been written covering various aspects of projects of this scope: books on project management and project development, on each part of the puzzle (web server, database, web site, web graphics, etc.), books and more books. Instead, this is a look at the project documentation from a design perspective -- what information is needed and expected, and the many ways information is represented to different teams on the project. The scope of the product, the market, and the budget dramatically affect how close this particular view is to a given project, but it's as good a place as any to start.

This not a road map, it's rather more like a travelogue or a series of snapshots. I think it's all but impossible to define a specific strategy for development, or create a definitive list of required project documents; every single project is new and different. For every rule one tries to define about how to conduct a project, there are 10 reasons to break that rule. The only strategy I've found effective is to recognize that there is no end product when it comes to electronic media, and there is no one winning strategy -- every product is but an artifact of the ongoing processes of discovery. It's extremely important to make certain to lay the groundwork for the expected and the unexpected transitions along the way - from one phase to another, from one architecture to another, from one software package to another, from one version to the next, from the old goals to the new ones, from the prototype to the launch vehicle.... Realize that you're participating in a recurring process, and retain something from each iteration, and with each iteration you can improve your understanding and your next strategy.

There are some basic "good habits," such as using a modular design to allow functional flexibility, and using the basic techniques of inheritance, allowing changes to propagate themselves wherever they are needed. "Extreme Programming" is a somewhat new idea in the development world that has applicability to the any design process. Working tests are pre-defined based on project requirements, then small teams work to create code that successfully passes those tests, and that process is reiterated ad infinitum. In other words, start by defining what "working" means, get something that meets that definition, then continue to keep it working indefinitely.

Finally, in regards the order of the illustrations below, if room permitted, these snapshots would perhaps be better arranged in a circle; these steps are not per se linear. Although some things must occur before or are directly related to others, many of these steps are independent and can be pursued in parallel.

 

filling in the blanks

	One fundamental change in thinking that the maturing computer industry has brought is a new kind of product cycle - the information cycle. During the industrial revolution, we learned to break a physical product into its component pieces, and mechanize the production of those pieces, culminating in an assembly line - Henry Ford and the Model T, 'nuff said. A common mistake in thinking about information and software products (for which a web site may be the interface) is to view them like a Model T - a thing that is made. The real "product" is the process itself - the assembly line, not the cars. This is a fundamental shift in perspective, in how we think and how that thinking affects how we work. In fact, it's not an assembly line at all, it's an "assembly network" (to coin a phrase), with many processes and teams functioning in parallel, designing and building multiple versions concurrently.
	Building an electronic community in some ways parallels the growth of a city - first comes a simple shelter (a page), then a house (site), a building (an application), a skyscraper (a local network), and finally a city (many networks in concert). As above, the key difference is that software serves a process that changes dynamically and rapidly - new versions are launched every 6 months, whereas cities take decades and centuries to evolve. Because information can be copied and reused in a way physical assets cannot, many process steps can now occur in parallel, making growth much more rapid. Every big site is a machine that gobbles up information from the company, and feeds it out endlessly to the consumers. The end result of a job well done is a well running assembly network that gathers and formats information, and readily disperses it to the "web site" (itself a database-driven internet application). It needs to cover every aspect of what a business does, and allow for the myriad people involved, including those behind the scenes, and those standing in line out front.
	The new "information cycle" is an endless spiral, germinating with one or more new ideas, gathering people and technology are those ideas, and with luck, spinning onward forever. The three primary creation processes (design, development and operation) occur in parallel. Software and content are continually modified and updated, feeding new iterations of design. By digitizing the the production cycle, and breaking it into its component pieces, new versions can be designed while previous products are still in operation, with at least one current version always being implemented. An extreme example of this sort of parallel development cycle is Intel, which has (at least) 3 major design teams working on 3 successive chip designs, with unrelated but parallel research going on for new production techniques that would facilitate completely new chip design techniques, while several completed designs are in production at all times. They are designing 3-10 years ahead of their current production, in part because of market competitiveness, in part due to the enormous time and cost involved in starting new production lines, but also because of the enormous efficiencies in growth created by designing many successive product versions in parallel.
project overview	The first step in any project or process is for those involved to work together to establish a set of goals and expectations. The business model must explored and understood, the market and audience are assessed, resources are considered, and a general project plan is put together. The attached represents a fairly comprehensive outline of the steps that might be taken over the course of the project, and a working list of the expected tasks.
standards	Establishing shared ground rules for working together is critical to the success of any project. The only hard and fast rule one can really make about conducting a project and establishing a process is in fact to agree to make ground rules together before you begin. This page is a working directory structure used to facilitate the design and development of a prototype application -- it offers an initial organization, a "place to put everything" and for everyone to store their work.
	"I know it's what I asked for, but it's not what I wanted." Oh, for the proverbial nickel every time I heard that. One of the biggest mistakes made in working relationships, whether between company and client, or between teams, is to assume that everything will go exactly as planned. Although you cannot know what the unknowns are, nor how the goals, expectations and limitations will change over the course of time, you can allow for change. My personal guideline is to establish the time and cost of what's been requested, and then double that estimate -- the worst case scenario is that everything goes perfectly, and the work is done in half the time at half the cost anticipated. Conversely, setbacks, new requirements and shifting goals don't affect the final deadline or budget (up to a point), because you've allowed enough elbow room in the schedule. Every project is different, and many things affect how much leeway to allow, but always always always budget for the unexpected.
functionals	As a friend once said, "forget you know anything, take on an anthropologist's/beginner's mind, and start by living with the people you are trying to serve." The first step in every project is to work with all participants to gather as much information on the problems, issues and goals as possible. No matter how unrelated or remote, it pays to explore every possible permutation of the process so that you can clearly define (or work with someone to define) the requirements. Eliminate the impossible, and whatever is left, no matter how improbable, must be the truth. The working result is a set of functional requirements -- a declaration of information required by every facet and at each step of the resultant ongoing processes.
	Coupled with the functional requirements is a sequence document -- what information must be gathered first, and what information depends on or requires previously gathered data. Every major interaction on this particular project had a corresponding functional and sequence diagram. This sequence helps the database designer establish interdependencies, the programmer establish the boundaries of what's acceptable as a request or response from a user, and the interface designer to correctly request and hand off data to the back end program.
	In this case, the company was a startup, so a corporate logo and corresponding color palette were created through an iterative process. The logo was designed to be acceptable in black & white as well as color, and to work on the web, in print, and on screen to effectively convey the company's identity.
	In concert with the logo, Pantone (printing ink) colors were chosen and the logo was integrated into business cards, letterhead, and web-compatible formats. The goal was (and should be) to present a consistent image across multiple media in variety of formats - again, establishing a baseline from which all visual design teams can work.
	A true web interface must provide frameworks for different viewers, and for disparate paths and tasks. This involves building individual pages, and creating a context within which to see these pages. Based on the project analysis, the functional requirements, and the company identity, all the pages of the site are mapped out, including their relative positions, their content source, navigational options, and database access points. One good test for an architecture is to ask the questions that the functional requirements pose, and trace the path to answers; a good architecture reuses as many resources as possible (the same page occurs many times), has short pathways to any given goal, and answers all expected requests.
	Object modeling is used to identify the entities that will be involved in an ongoing way with gathering and delivering information (the people within the company, and the clients it serves), as well as the functional relationships between those entities. These eight objects provide the conceptual basis for creating a working application; there are organizations, as well as users which have roles, either within an organization, or in relationship to it (e.g. customers). The application has the ability to provide secure communications through RSA encryption. Each entity has a session (~ particular occurrences), which has a location -- for instance, a user has provided a specific address for shipping on this particular order. These objects and relationships form a core, generic model of how a business and its customers relate, and allow corresponding software to be written, and matching data to be stored in a database. The model can also inform the site architecture and flow. Such a model serves to provide a visual keystone to all the design teams as the process moves forward.
	A class hierarchy is sometimes used to define the how complex software functions needed for a particular process and set of processes will be built up through increasing layers of specificity. There are two ways to arrive at any hierarchy: from the bottom up, by identifying all the things that will need to be done, and then factoring out common programming tasks; or from the top down, by identifying the simplest and most generic tasks first, and obtaining ever-increasing complexity by building upon each succeeding set of functions.
	Virtually if not all modern databases are relational in nature. They are composed of individual tables that have fields and values (name=jones, ssn=123.12.1234, etc.), and relationships are specified between tables. There is a direct (though not necessarily one-to-one) relationship with defined objects and classes.
	In this particular model, the core table is probably the entity table -- this corresponds directly to the Entity object in the object model. Every object / entity that is created (for a new user, or a new company, or a new address) has its own unique entity ID. Tables have required relationships (solid lines -- a person must have an entity ID), as well as optional relationships (dotted lines -- an entity can have a type). Database design is a highly specialized field of Information Design; a good database design must not only store all functionally required data, but should do so in a minimum of space, and in such a way as to maximize efficient recovery of information -- speed is critical. The database server and working database design are frequently the single most expensive part of a large scale development effort.
	In order to deploy a network application, to launch a web site, start a company or retrofit an existing business unit, a number of different computers must be integrated into a local network. The network architecture must provide access points for the primary company, the clients and customers, and the general public, and can have additional ties to support systems, such as other on line databases, the banking industry or a credit card processing firm. Different users have access to different layers of information, depending on their particular set of permissions. For instance, only developers have the ability to modify software on primary and development machines. Web users should have extremely limited access to only the pages of the web site. Frequently, as in this illustration, there are multiple networks, including public access (via the web) and private access (via a securely encrypted channel only).
	Hardware specifications are made based on the project analysis & requirements, including expected usage, complexity of the software, operating systems and versions, security requirements, expected growth, budget and available technology. All too often I see projects that skimp on hardware, believing it to be expensive. The truth is that hardware is already obsolete, forever expendable, and in the end, it is a productivity tool for people. The better the tools, the better, faster and more efficiently people can inter-operate.
	Many projects seem to leap immediately into designing the finished web site, in part because the interface is the most tangible evidence of the work that everyone does (it's what facilitates the tactile experience of every user after all). Still, I think that the web interface properly belongs nearly at the end of all other work; it is the culmination of all the other information design and project organization. The interface relies heavily on work that goes before, including functional requirements, sequence diagrams, object models, corporate identity and business analysis.