In Part 1 of our look at the Sedona Conference’s Commentary on Achieving Quality in the E-Discovery Process we looked at how the application of project management processes could help define and control the E-discovery process. In Part 2, we want to again look in detail at what the commentary has to say about measuring quality, and why this is vital to the success of your process. 

In the first section of the commentary we notice that the Sedona Conference makes an effort to take the common business practices regarding project management and apply them to E-Discovery. In this section, they take another common business concept, quality measurement, and discuss why it’s important that we learn to measure the quality of our discovery efforts. 

The commentary lays out four reasons why quality is important. Poorly done E-discovery processes can miss key evidence, could include privileged documents inadvertently, isn’t defensible, and can be more costly when you have to go back and do it again. All of these are valid reasons to be considering quality as you go through your E-discovery process. 

Sedona also has a recommendation about quality control, versus quality assurance. In their view quality control, meaning that “quality” is engineered into the process up front, is superior, and more feasible in most cases, compared to quality assurance, which involves looking back over the process at its conclusion. In the context of litigation it is easy to see why taking steps to get it right up front makes much more sense than a post-hoc review that has only downside potential for the client. After the fact, CQI-type programs carry a special risk of being discovered and exploited by current and future litigation adversaries. 

In a nutshell, the benefit of measuring quality while the discovery process is on-going is to enhance the defensibility of your processes, should they later be challenged, while informing lead counsel of corrective action that can be taken to mitigate any problems before they show up in a motion for sanctions . There should be, at minimum, someone checking to make sure your search results make sense, that the files you’re putting into a piece of technology to be reviewed are also coming out of that system for production, and that the reviewers and coders are doing what you expect as you review documents. There’s nothing worse than spending days, weeks or months with a team of reviewers only to find out that not all of them were using the same taxonomy to identify privileged documents, causing your identification of those documents to be incorrect. 

As a practical matter, having quality control built in to the process allows you to catch problems when they happen, and correct for them, rather than having to redesign an entire process and start from scratch each time. That can be a significant savings of time and expense. 

Sedona identifies five measures of quality:

• Judgmental sampling
• Independent testing
• Reconciliation techniques
• Inspection to verify and report discrepancies
• Statistical sampling

Those with an accounting or engineering background are no doubt familiar with these sorts of measures and their role in measuring quality at varying points in a process. The commentary admits that there are a number of barriers to the adoption of these sorts of quality measures to E-Discovery. Differences in particular cases make a single-cookie cutter approach impractical for all cases, fueling confusion as to what a reliable quality plan must entail. “Reasonableness” remains the bearing point. Perfection is neither required nor economically feasible. A sensible, well-informed application of these quality measures commensurate with the needs of a particular case can help a party comply with their obligations.

You can download your own copy of the Commentary from the Sedona website.

In Part 3 we will look at specific ways the commentary talks about applying quality measures in E-discovery.

The recent publication from the Sedona Conference, Commentary on Achieving Quality in the E-Discovery Process, released in May 2009, attempts to bring the concept of quality control, as applied in many areas of business organizations, and show how these same processes can be used to help manage the E-discovery process. 

In the opening paragraph of the introduction, the commentary describes the legal profession as being at a crossroads, between handling discovery the way it always has, or recognizing that the growth of electronically stored information is fueling the need for new processes and technical approaches to discovery. The goal is to introduce some elements of “quality measures” that may assist in “taming the ESI beast”. 

After the Introduction, the commentary is broken into two parts, Achieving Quality Through Project Management and Better Measurement, and secondly Applying Quality Measures in E-discovery. Today, we’ll look only at the first half of part one. 

The Importance of Project Management

This section of the commentary starts off with what should be a no-brainer to those of us working every day in an IT environment, explaining why any E-discovery project should have a well-defined process. Obviously, any time you look at even the simplest document review, let alone more complex E-discovery issues, you define the process before you even start. You define the goals; who is in charge, how you will measure success, and what the final results should look like. That over-reaching process lets everyone know what they should be doing, what still needs to be done and what kind of deadlines you’re up against. 

They then go on to describe seven key process elements that, depending on the scope of your project, may need to be in place to lay the groundwork for the successful application of technology to the problem at hand: 

1. Leadership. The process should be led by a person who will be responsible for assuring that a discovery process reflects a reasonable good-faith effort to be complete and accurate.

2. Tailoring. The discovery process should be tailored to the specific size, risks, needs and circumstances of the case or investigation that is the occasion for the retrieval effort.

3. Expertise. The effort should incorporate and draw on the appropriate range of expertise required to meet and accomplish the goals set for it in a timely and cost-effective manner.

4. Adaptability. The process should be iterative and adaptive, allowing for learning and course correction as the project unfolds.

5. Measurement. Where appropriate, elements of the process should employ metrics in order to monitor the progress of the effort and to ensure consistent quality of results.

6. Documentation. The overall process should be documented to ensure coordination and communication within the discovery team and to increase the defensibility of the process.

7. Transparency. The selection, design, implementation, and measurement of a process should be able to be explained in a clear and comprehensive way to the relevant fact-finder, decision-maker, tribunal, or regulator, as well as to opposing counsel as may be appropriate.

I found it interesting that they chose “Leadership” as the very first element. In fact, there’s quite a bit of discussion in the next sub-section about the need for a team leader and who that person might be depending, again, on the details of your case. Inherent in this discussion of team leader is the understanding that it does take a team to do E-Discovery well. It’s not just the job of counsel, but it takes expertise in many areas and requires the involvement of a legal team (whether that is inside counsel, outside counsel, or some combination), the custodians of ESI, the IT team, and potentially technology expertise that is brought in by litigation support, vendors, or consultants. Obviously, the next two elements work hand in hand with this team concept, tailoring the approach, and the team, to the specifics of the case, and including the proper expertise on the team. 

The other two key process elements I want to talk about in more detail are the last two, “Documentation” and “Transparency.” These elements of project management make it much easier for the team lead, or lead counsel in most cases, to answer any and all challenges to the E-discovery process. Having team members document what they do, and how they do it, ensures that the team leader can verify the results, by retracing the steps involved if need be. It also enables better communication among team members, eliminating duplicate work or unneeded delays. 

Having transparency built in to the process from the ground up, which includes good documentation of course, allows for all the coordination between team members to go smoothly, but also ensures that, when challenged, it will be easy for counsel to provide the necessary explanations and measurements to defend the process. It also can help with the cooperative nature of E-discovery between the parties by making it easy for opponents to see that, yes, indeed, the process was reasonable and complete, letting them move past the “beast” of E-discovery and on to trying the case on it’s merits. 

The Sedona Conference is correct in their opening paragraph. ESI is not the same as paper. Locating, reviewing, and producing relevant ESI can be a highly complex task, involving any number of resources, from human labor and time, right through equipment and technology tools. Proper coordination and scheduling of those resources can help the process be both smooth and defendable, and that sure sounds a lot like project management to me.

You can download your own copy of the Commentary from the Sedona website.

In Part 2 we will look at the need to measure quality in E-discovery.

Legal Problems of “Links”
When you use a link, your computer contacts that address and downloads for your viewing a copy of the web page it finds there. So, as a result of the technology, you are in a sense "copying something" as you surf the web. That technical convenience has created some of the legal issues regarding web linking that relate to copyright and trademark infringement. Those legal issues are extensive and complicated, and are very tied to the individual facts of a certain use - and are not the subject of this comment. 

As with any 'publication', a web site can also be used to defame or libel, interfere with business relationships, or do other things that cause damage to persons that can be the subject of a lawsuit. There are specific Federal and State laws that regulate web activity, particularly as it regards children, and these subjects are likewise beyond this comment. What we will focus on is whether or not "free speech" rights under our Constitution apply to create a citizen's right to have his or her web page linked on a government web site. 

Free Speech has Limits
Among the protections of our freedom found in the First Amendment to the US Constitution ("Am. 1") is the right of "free speech". That right covers not only the ability of someone to speak, but also the right of citizens to access certain places and media to make their opinions known to others. For example, in Colonial times, the so called "pamphleteers" were protected in passing out anonymous flyers criticizing the government. In our time, we have seen the requirement for "equal time" on radio and television for political parties and candidates. So, the right to a "forum", i.e., a place or a medium to air opinions, is protected under our Constitution.

As with most rights, there are limits and responsibilities that go along with the right to free speech. Courts over the years have recognized that there are different places (forums) where people might like to air their opinions, and so there are different rules for the different places. Obviously, ownership has something to do with that analysis - you are not, for example, required to allow someone to make a speech in your backyard! However, another part to the analysis is precedent - whether or not you have allowed people to use your property or media for the expression of opinions. 

When it is the government that owns the property or the media (such as a website), these principles still apply – but because the public is the ultimate “owner”, the reasons must be clearer and the analysis more careful before a member of the public is denied access. Because government agencies and political subdivisions are ultimately owned by the public, they must be very careful to understand and provide for the rights of citizens, while fully understanding their own rights and their limits. 

Various “Forums” exist for Free Speech
A “public forum”, such as a public park, is hard to close to public speech. Courts have held that only reasonable “time, place and manner” restrictions can be imposed, and persons can never be excluded from public forums on the content of their speech. The reasonable restrictions that are allowable are for public safety and health, and so that the speakers do not impinge on the rights of others – such as by playing loud music in the park during sleeping hours (Ward v. Rock against Racism, 491 U.S. 781 (1989))

Sometimes, government agencies will open a forum for speech on a “semi-public” basis, which in law is called a “limited” forum. A good example is allowing the use of an auditorium in a government building when it is not being used for regular business. In many ways, this limited forum distinction is what is codified in Ohio law in allowing the use of school buildings for community events, etc., when it does not interfere with the operation of the school. (O.R.C. 3313.76 and 3313.77) This type of access can be more restrictive, in that those in charge of the facilities may make regulations, specify times, require insurance, and other limitation, but the access must still be largely “content neutral”. This is because the government agency knowingly and purposefully opened the forum for public use. 

There Are NON-Public Forums
The final “category” is property or media that are not public forums. There are places in government property and media controlled by the government that have never been opened to the public for their use. In most cases, to open such places or media would have a detrimental influence on getting the mission accomplished that the government agency exists to carry out. If the government has a “track record” of not allowing public access to a certain place or medium, and if that restriction is reasonably related to the government purpose, then the public can be excluded. 

City Web Page; Public Forum or Not? The Putnam Pit Case
This principle went to court, as it applies to web pages owned by government entities, in a case entitled Putnam Pit, Inc. v. City of Cookeville. (76 Fed. Appx. 607 (6th Cir. Tenn. 2003), 2003 U.S. App. LEXIS 17775. The very brief facts are that the owner of the “Putnam Pit”, a “newspaper” and web site dedicated to finding and publicizing scandalous news about the government, requested that a link to their website be placed on the official City of Cookeville page. The City denied the request, stating that "the standard for web links was that [the linked website] must promote the economic welfare, industry, commerce, and tourism in the local area to be linked to the web site." Prior to this request, the City had been allowing private businesses to have a link on the page, and the Mayor stopped that practice as well – as part of demonstrating that the website had a limited purpose. 

The Court held that the City could exclude the Putnam Pit from its website, in these words:

“The city of Cookeville's policy, in addition to being reasonable in light of the city's interest, must also be viewpoint neutral. Although the avoidance of controversy is not a valid ground for restricting speech in a public forum, a nonpublic forum by definition is not dedicated to general debate or the free exchange of ideas. The First Amendment does not forbid a viewpoint-neutral exclusion of speakers who would disrupt a nonpublic forum and hinder its effectiveness for its intended purpose. Therefore, while the city may restrict use to those who participate in the forum's official business, it may not do so based on viewpoint. In other words, (Putnam Pit) has no entitlement to a link to the city's Web site, however, he may not be denied one solely based on the controversial views he espouses, without regard for the forum's purpose and structure.” Id. at 612 (emphasis added)

So, as said before, the critical exception that allows a government agency to exclude a certain web link is when that link is not in keeping with the purpose and structure of what the government agency is trying to accomplish by means of the website. That is what a school district will have to show in court to support their denial of web linkage to a requestor. In this case, the City was able to show that regardless of the Putnam Pit’s extreme views, it was their lack of relevance to the government’s mission and not their viewpoint that made their link unsuitable for the page. 

Some Things You Can Do; Policies and Procedures
So – what can a public entity do to properly prepare to handle such requests? 

First, it IS possible to make a policy that the government entity does not link to ANY outside site. That is, of course, the safest route to avoid challenges, but it eliminates a lot of the ease of use for a user who can otherwise speedily find relevant links. 

A more reasonable approach is something like what the City of Crooksville decided; to make a policy that states what the purpose of the website is, how it is intended to support the mission of the district, and what constraints will be placed on all content and all web links. This should start with a statement in Board policy, at the highest level, and be carried through all of the entity’s acceptable use policies, instructions on web authorship, and even be repeated on the website itself. 

An important thing to show a court, if a requestor complains about not receiving a web link on the government page, is that the government entity never opened the web site to public opinion or public use. This says to the court that the entity did not intend that a public or semi-public forum be created. The government entity can then go on to show the court that web links are placed on the pages because of their “fit” with the mission stated in policy, and NOT because of a determination about their political or other viewpoint. 

A final method of protection is to have a disclaimer statement either on the links page, or as a “clickthrough” that pops up before the user is taken to an outside site. This is a good idea on multiple levels, not just as a rationale against outside requests. Web pages are capable of instant and constant revision, and it is entirely possible for highly objectionable material to replace acceptable content without the government entity’s knowledge. 

Such disclaimers, as in these samples from school districts, might say: 

The district is not responsible for the content of outside links.
Linked sites are not under the control of the school district, its agents or employees. The school district is not responsible for the contents of any linked site, any link contained in a linked site or any changes or updates to such sites. The school district provides links as a convenience, and the inclusion of any link does not imply endorsement of the site by the school district. The school district reserves the right to restrict any links to sites containing inappropriate, obscene, sexually explicit, or other material that is inconsistent with the mission of the school district or outside the scope of permissible uses as articulated in board policy

Or, the same information can be bulleted for better graphic look:

You are about to leave the Official _______ Local School District Website.If you choose to follow this link, please note that:

• The views and information or opinions expressed on this web site do not necessarily represent the views of the ________Local School District.
• The District does not control and cannot guarantee the relevance, timeliness or accuracy of the materials on the linked sites
• The link is provided for user convenience, and the link does not imply District endorsement or approval of the material or the responsible party

Board or agency policy should contain (at a minimum) this sentence, in the general policy covering the website:

Because the District is not opening a public or limited public forum by providing web services, the District retains the right, in the sole discretion of designated personnel, to control the content of the web pages it hosts.

And finally, guidelines to web authors should contain something like the following, as should the acceptable use policy:

Subject Matter — All subject matter on School District Web pages and their links must relate to curriculum and instruction, school-authorized activities, or information about the District or its mission. The district website is not open as a forum for public opinion. Staff or student work may be published only as it relates to a class project, course, or other school-related activity. Students, staff, or other individuals may not use the district's web pages or links to provide access to their personal pages on other servers, online services, or to any site not deemed consistent with the District’s mission.

If you desire more information on this topic, the following links may be helpful:

• http://www.linksandlaw.com/ (a good general site)
• http://www.llrx.com/features/internetwaves.htm (specifically about public website links)
• http://www.linksandlaw.com/decisions-14.htm (the Court’s decision in “Putnam Pit” – the case referenced in this article)
• http://en.wikipedia.org/wiki/Hyperlink (Wikipedia article on linking; broad overview but includes legal aspects)
• http://www.cybertelecom.org/ip/link.htm (copyright/trademark legal aspects of linking)