7 Things E-Discovery Auditors Must Do

U.S. Federal Rules of Civil Procedure (FRCP) require organizations to look at the ability to respond in a legally defensible manner to discovery requests. Moreover, as organizations expand globally, they need to be ready at all times to provide information that could be requested as evidence in a legal proceeding. Internal or external auditors are in the best position to recommend policies and best practices that can prepare organizations to respond to a data discovery request. The auditors must:

  1. Determine the effectiveness of the e-Discovery communication plan

  2. Document the IT environment

  3. Regularly review backup, retention, and data destruction policies

  4. Review compliance with document destruction procedures, when a litigation hold is issued

  5. Document the steps that will be taken to respond to e-discovery requests

  6. During litigation, determine whether employees are preserving the integrity of relevant material

  7. Review existing backup controls, reports, and inventories of media stored off site

Failing to prepare for an e-discovery request can result in sanctions. Organizations need to have a litigation readiness policy and plan in place to effectively deal with lawsuits. Auditors play a pivotal role in managing litigation risks and help organizations take a proactive approach to e-Discovery by recommending strategies that address key data preservation, storage, destruction, and recovery disquiets. Microsoft SharePoint 2013 and M-Files, for instance, offer e-Discovery and content management solutions to cater to these needs.

e-Discovery auditor

e-Discovery auditor

e-Discovery and | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043


Corporate Social Responsibility (CSR) in E-Discovery Industry

Triple Bottom Line

Corporate Social Responsibility (CSR) is a management concept in which companies integrate social and environmental concerns in their business operations and interactions with their stakeholders. The basic definition according to Wiki:

“Corporate social responsibility is a form of corporate self-regulation integrated into a business model. CSR policy functions as a built-in, self-regulating mechanism whereby a business monitors and ensures its active compliance with the spirit of the law, ethical standards, and international norms”

CSR is best incorporated with the ‘Triple Bottom Line’ (TBL) approach, which is essentially an accounting framework incorporating three dimensions of performance: financial, social, and environmental.

Triple Bottom Line

Triple Bottom Line

A triple bottom line measures the company’s economic value, ‘people account’ – which measures the company’s degree of social responsibility and the company’s ‘planet account’ – which measures the company’s environmental responsibility. While CSR indoctrination within the e-Discovery industry may be prevalent, only a handful of companies may actually have developed and adopted CSR.

Adopting to a mindset of a good corporate citizen, at ClayDesk, we have initiated the development of a CSR program with a goal of embedding CSR practices in our business. The foremost area of focus for CSR initiatives are directed towards promotion of legal education, e-Discovery laws, Pro-Bono legal work, sponsor a student, and steps towards a paperless (go-green) environment. These steps will bring about positive change and improve the quality of life of members of the society.

Some of the core CSR issues relate to: environmental management, eco-efficiency, responsible sourcing, stakeholder engagement, labor standards and working conditions, employee and community relations, social equity, gender balance, human rights, good governance, and anti-corruption measures. Denmark, for instance, has CSR Law in place which mandates companies to report their CSR initiatives. Apart from providing charity and sponsorship, CSR concept goes beyond by allowing companies the opportunity to become a socially and ethically responsible corporate citizen.


ClayDesk’s committment to CSR

e-Discovery and | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043

When Should E-Discovery Vendors Be Disqualified? Gordon V. Kaleida Health Case

Generally speaking, courts have inherent authority to disqualify parties, representatives, and consultants from participating in litigation.  Attorneys, expert witnesses, and litigation consultants may face disqualification motions in the event of a conflict of interest. With the rapid expansion of the eDiscovery industry, however, a new question has arisen: If an eDiscovery vendor has a potential conflict of interest, when should it be disqualified?  What standard should apply?

To put the problem in perspective, imagine that you manage discovery at a law firm representing the defendant in a contentious wage and hour dispute, and you recently hired an eDiscovery vendor to assist you in scanning and coding your client’s documents, at a cost of $50,000.  Two months later, you receive notice from your vendor that the plaintiff’s counsel has requested its services in connection with the same case.  How would you react?  Would you expect a court to disqualify the vendor if it accepted the engagement?  This scenario occurred in Gordon v. Kaleida Health, resulting in the first judicial order squarely addressing vendor disqualification.  The Kaleida Health court ultimately denied the defendant’s motion to disqualify, allowing the vendor to continue participating in the case.

Discussion of Gordon v. Kaleida Health

Kaleida Health arose out of a now commonplace dispute between a hospital and its hourly employees under the Fair Labor Standards Act (“FLSA”). The plaintiffs, a group of hourly employees, sued the defendant, Kaleida Health, a regional hospital system, claiming they were not paid for work time during meal breaks, shift preparation, and required training, in violation of FLSA.

Kaleida Health’s attorneys, Nixon Peabody, LLP (“Nixon”), hired D4 Discovery (“D4”), an eDiscovery vendor, to scan and code documents for use in the litigation. In connection with the work, Nixon and D4 executed a confidentiality agreement. D4 was to “objectively code” the documents using categories based on characteristics of the document, such as the author and the type of document. The coded documents would then be used by Nixon in preparing for upcoming depositions.

Two months later, plaintiffs’ counsel, Thomas & Solomon, LLP (“Thomas”), requested D4 to provide ESI consulting services to it in connection with the same case. D4 notified Nixon, who promptly objected based on the scanning and coding services D4 provided the defendant during the litigation. D4 then provided assurances that Kaleida Health’s documents would not be used in consulting the plaintiffs and that an entirely different group of employees would work with the plaintiffs’ counsel. Nixon, on behalf of Kaleida Health, persisted in its objection to D4 working for the plaintiffs and ultimately filed a motion to disqualify the vendor.

Magistrate Judge Foschio’s analysis began by outlining the standard governing the disqualification of experts and consultants.  According to the court, the entity sought to be disqualified must be an expert or a consultant, defined as a “‘source of information and opinions in technical, scientific, medical or other fields of knowledge’” or “one who gives professional advice or services” in that field. After the moving party makes this initial showing, it must meet two further requirements.  First, the party’s counsel must have had an “‘objectively reasonable’ belief that a confidential relationship existed with the expert or consultant.” Second, the moving party must also show “that . . . confidential information was ‘actually disclosed’ to the expert or consultant.”

Applying this standard, Judge Foschio ultimately found that because the scanning and objective coding services performed by D4 did not require specialized knowledge or skill and were of a “clerical nature,” D4 was not an “expert” or “consultant.” Further, the court determined that the defendant failed to prove that it provided confidential information to D4 because it did not show “any direct connection between the scanning and coding work . . . and Defendants’ production of [its] ESI.”

Rejecting Kaleida Health’s argument, the court declined to apply to D4 and other eDiscovery vendors the presumption of confidential communications, imputation of shared confidences, and vicarious disqualification applicable in the context of attorney disqualification when a party “switches sides.” The court— as an alternative basis to its finding that D4 did not act as an expert or consultant—held that disqualification was improper because no “prior confidential relationship” existed between Kaleida Health and D4.

Because Kaleida Health represents the first significant attempt at exploring the issues surrounding vendor disqualification, whether later courts should follow Kaleida Health’s lead in exclusively applying the disqualification rules for experts and consultants to vendors becomes the main issue in its wake.  To come to a conclusion on this point, one must first explore the different schemes that courts may apply when considering disqualification.

This above excerpt is a part of article originally written by Michael A. Cottone, a candidate for Doctor of Jurisprudence, The University of Tennessee College of Law, May 2014.

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043

Appellate Court

Appellate Court – Lahore

The trade-off between ‘Recall’ and ‘Precision’ in predictive coding (part 2 of 2)

This is the second part of the two-part series of posts relating to information retrieval by applying predictive coding analysis, and details out the trade-off between Recall and Precision. For part 1 of 2, click here.

To clarify further:

Precision (P) is the fraction of retrieved documents that are relevant, where Precision = (number of relevant items retrieved/number of retrieved items) = P (relevant | retrieved)

Recall (R) is the fraction of relevant documents that are retrieved, where Recall = (number of relevant items retrieved/number of relevant items = P (retrieved | relevant)

Recall and Precision are inversely related. A solid criticism of these two metrics is the aspect of biasness, where certain record may be relevant to a person, may not be relevant to another.

So how do you gain optimal values for Recall and Precision in a TAR platform?

Let’s consider a simple scenario:

• A database contains 80 records on a particular topic

• A search was conducted on that topic and 60 records were retrieved.

• Of the 60 records retrieved, 45 were relevant.

Calculate the precision and recall.


Using the designations above:

• A = Number of relevant records retrieved,

• B = Number of relevant records not retrieved, and

• C = Number of irrelevant records retrieved.

In this example A = 45, B = 35 (80-45) and C = 15 (60-45).

Recall = (45 / (45 + 35)) * 100% => 45/80 * 100% = 56%

Precision = (45 / (45 + 15)) * 100% => 45/60 * 100% = 75%

So, essentially – the optimal result – high Recall with high Precision is difficult to achieve.

According to Cambridge University Press:

“The advantage of having the two numbers for precision and recall is that one is more important than the other in many circumstances. Typical web surfers would like every result on the first page to be relevant (high precision) but have not the slightest interest in knowing let alone looking at every document that is relevant. In contrast, various professional searchers such as paralegals and intelligence analysts are very concerned with trying to get as high recall as possible, and will tolerate fairly low precision results in order to get it. Individuals searching their hard disks are also often interested in high recall searches. Nevertheless, the two quantities clearly trade off against one another: you can always get a recall of 1 (but very low precision) by retrieving all documents for all queries! Recall is a non-decreasing function of the number of documents retrieved. On the other hand, in a good system, precision usually decreases as the number of documents retrieved is increased”

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043

Recall and Precision

Recall and Precision

The trade-off between ‘Recall’ and ‘Precision’ in predictive coding (part 1 of 2)

This is a two-part series of posts relating to information retrieval by applying predictive coding analysis, and details out the trade-off between Recall and Precision.

Predicting Coding – sometimes referred to as ‘Technology Assisted Review’ (TAR) is basically the integration of technology into human document review process. The two-fold benefit of using TAR is speeding up the review process and reducing costs. Sophisticated algorithms are utilized to produce relevant set of documents. The underlying process in TAR is based on concept of Statistics.

In TAR, a sample set of documents (seed-sets) are coded by subject matter experts, acting as the primary reference data to teach TAR machine recognition of relevant patterns in the larger data set. In simple terms, a ‘data sample’ is created based on chosen sampling strategies such as random, stratified, systematic, etc.

Remember, it is critical to ensure that seed-sets are prepared by subject matter experts. Based on seed-sets, the algorithm in TAR platform starts assigning predictions to the documents in the database. Through an iterative process, adjustments can be made on the fly to reach desired objectives. The two important metrics used to measure the efficacy of TAR are:

  1. Recall
  2. Precision

Recall is the fraction of the documents that are relevant to the query that are successfully retrieved, whereas, Precision is the fraction of retrieved documents that are relevant to the find. If the computer, in trying to identify relevant documents, identifies a set of 100,000 documents, and after human review, 75,000 out of the 100,000 are found to be relevant, the precision of that set is 75%.

In a given population of 200,000 documents, assume 30,000 documents are selected for review as the result of TAR. If 20,000 documents are ultimately found within the 30,000 to be responsive, the selected set has a 66% precision measure. But if another 5,000 relevant documents are found in the remaining 170,000 that were not selected for review, which means the set selected for review has a recall of 80% (20,000 / 25,000).

Click here to read part 2 of 2.

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043

CEO ClayDesk

Syed Raza

4 Ways SharePoint 2013 e-Discovery center can keep you ‘Litigation Ready’

The biggest cost driver in e-discovery is document review.  Law firms and corporations are actively seeking ‘ways’ to curtail costs without compromising on quality. Predictive coding technologies, outsourcing, offshoring, dual-shoring, insourcing, and other methods are all the ‘ways’ that can be utilized in order to meet desired objective – bring costs down!

How many of us truly think about being ‘litigation ready’ or being prepared for future litigation? Having a proactive approach is sometimes difficult, especially when costs are key a concern, however, it may actually result in being cost effective in the longer run. Microsoft SharePoint e-Discovery module is essentially the proactive part of maintaining litigation readiness. According to Microsoft:

“Typically, e-Discovery requires searching for documents, websites, and email messages spread across laptops, email servers, file servers, and other sources, and collecting and acting on content that meets the criteria for a legal case. In SharePoint Server 2010, Microsoft added the Hold and e-Discovery feature, which made it possible to place a hold on any site in SharePoint. A records manager could put documents, pages, and list items on hold, which prevented users from deleting or editing them. Exchange 2010 introduced a way to place legal holds on mailboxes, conduct searches across multiple mailboxes, and use a Windows PowerShell cmdlet to export mailboxes.”

E-Discovery in SharePoint 2013 includes new ways to reduce the cost and complexity of discovery. These include:

  • The e-Discovery Center, a central SharePoint site used to manage preservation, search, and export of content stored in Exchange and SharePoint across SharePoint farms and Exchange servers.
  • SharePoint In-Place Hold, which preserves entire SharePoint sites. In-Place Hold protects all documents, pages, and list items within the site but allows users to continue to edit and delete preserved content.
  • Exchange In-Place Hold, which preserves Exchange mailboxes. In-Place Hold protects all mailbox content through the same UI and APIs used to preserve SharePoint sites.
  • Query-based preservation allows users to apply query filters to one or more Exchange mailboxes and SharePoint sites and restrict the content that is held.

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043

SharePoint 2013 e-Discovery

SharePoint 2013

Outsourcing or Insourcing? The Balanced Score Card approach

Outsourcing is now a common phenomena among businesses. Essentially, you contract out a business process to a third party, both foreign and domestic contracting – at times relocating a business function to another country. Traditionally speaking, companies having financial difficulties didn’t have much choice but to restructure, lay-off employees or incur additional debt to cover short term obligations. Outsourcing of redundant business processes came as a sigh of relief for many, especially large corporations with humongous overheads. The incentive to outsource may be greater for U.S. companies due to unusually high corporate taxes and mandated benefits such as Social Security, Medicare, and Occupational Safety and Health Administration (OSHA) regulations.

Now that you have successfully outsourced and established a cordial relationship with your vendor, industry trends may change that require you to focus more on certain business segments.  Insourcing has been identified as a means to ensure control, compliance and to gain competitive differentiation through vertical integration or the development of shared services (commonly called a ‘center of excellence’).

But wait a minute! It seems like you require both in order to attain a lean and optimal cost effectiveness business model.

E-Discovery Industry:

For the last decade or so, e-Discovery industry has experienced tremendous growth in terms of outsourcing mainly due to escalating costs. The result – growing number of captive centers all over the world, especially with abundant labor resources such as India, Pakistan, and Philippines. The biggest cost driver in e-Discovery is document review – over 60 percent of total costs, one of the main reasons of influx of captive offshore centers or outsourcing. Recently, however, law school graduates in the US, for instance, have been accepting employment at all time low wages – making it increasingly competitive for e-Discovery vendors.

What’s an optimal solution?

According to Balanced Score Card Institute, “The balanced scorecard is a strategic planning and management system that is used extensively in business and industry, government, and nonprofit organizations worldwide to align business activities to the vision and strategy of the organization, improve internal and external communications, and monitor organization performance against strategic goals” Using this tool helps an organization to identify, understand, and evaluate core business processes, resulting in best practices of utilizing outsourcing and insourcing strategies.

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043


Can ‘Predictive Coding’ cope up with ‘Big Data?


Can ‘Predictive Coding’ cope up with ‘Big Data?

Discovery has changed, and electronically stored information (ESI) was the facilitator. Though ediscovery matters are no longer the novel issues that they once were,” technology is constantly changing. According to Baseline, it was estimated that 90 percent of worlds data has been created in the last two years.  in 2009 there were 988 Exabyte of data in existence, an amount that would stretch from the Sun to Pluto and back in paper form. The problem for corporations is the storage of huge amounts of data – let alone worry about the ‘compliance monster’.

Perhaps, cloud computing is here to ease things out, yet companies are retaining more information than ever, and lawsuits sometimes require attorneys review millions and millions of documents. While Judiciary struggles to devise effective mechanism regarding proportionality rules, big data is growing even bigger – not to mention growing litigation industry. It seems manual review of documents is not an option anymore, as technology is rushing towards meeting the growing needs of document review.

The most important element overlooked is the fact that human eyeballs are still required to review such documents leading to defensibility of the case; after all, isn’t that the real objective?

Definitions of “predictive coding” vary, but a common form of predictive coding includes the following steps. First, the data is uploaded onto a vendor’s servers. Next, representative samples of the electronic documents are identified. These “seed sets” can be created by counsel familiar with the issues, by the predictive coding software, or both. Counsel then review the seed sets and code each document for responsiveness or other attributes, such as privilege or confidentiality. The predictive coding system analyzes this input and creates a new “training set” reflecting the system’s determinations of responsiveness. Counsel then “train” the computer by evaluating where their decisions differ from the computers and then making appropriate adjustments regarding how the computer will analyze future documents.

This process is repeated until the system’s output is deemed reliable. Reliability is determined by statistical methods that measure recall—the percentage of responsive documents in the entire data set that the computer has located—and precision—the percentage of documents within the computer’s output set that are actually responsive. (That is, “recall” tests the extent to which the predictive coding system misses responsive documents, while “precision” tests the extent to which the system is mixing irrelevant documents in with the production set.) The resulting output can be either produced as is or further refined by subsequent human review. Subsequently, attorneys review a much smaller set of documents. Predictive coding therefore effectively “alleviates the need to review whole masses of records in order to find the relevant few.” Most importantly, predictive coding is estimated to reduce ediscovery costs as much as 40% to 60% while maintaining search quality.

A statistic quoted in an IDC and EMC report says that the digital universe is doubling every two years, and will reach 40,000 Exabyte (40 trillion gigabytes) by 2020. The question is: Can predictive coding cope up with big data?

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
info www.claydesk.com
(855) – 833 – 7775 (703) – 646 – 3043

Managed Review vs. Staffing Model

There are two standing industry models for outsourcing e-discovery document review projects – the managed review and the staffing model. While selecting a review solution – either staffing of managed, the solution should reflect an approach suffused in an understanding of applied best practices. Having said that, the following sets forth a minimal, standardized, framework which can and should be adapted to meet the needs of specific cases.

Project management

· Ensure a project plan is specifically designed and crafted to the specifications and requirements of counsel

· consistent with best practices

· Deliver a key set of documents or review protocol that govern the execution and project

· management of the review process along with workflow design

Team selection and training

· Staff personnel with expertise in specific area of laws relating to project. Develop specific job descriptions and define a detailed protocol for recruiting, testing, and selection

· Ensure conducive environment as well as conduct reference and background checks

· Preferably engage experienced personnel

· Ensure the review team receives comprehensive substantive and platform training


· Design processes, assignments and quality assurance steps specifically tailored to the project’s requirements

· Demonstrate compliance with key security and quality standards while maintaining acceptable pace

Quality control

· Adhere to six sigma principles

· Develop effective quality control processes to achieve key project goals

· Test first review work product using sampling method, targeted re-review, and validations searches

· Conduct statistics to ensure the highest quality end result

· Maintain auditing and track performance of reviewers


· Develop a formal schedule of communications with counsel adhering to laid down schedules

· Calibrate initial review results, seeking counsel’s guidance to confirm or correct results and to conform review protocol and training materials to insights gained


· Deliver regular, comprehensive reports to monitor progress and quality and to assist counsel in managing the review process

Productions and Privilege Logs

· Prepare privilege logs in accordance with specifications set by counsel

e-Discovery | cloud computing
New Jersey, USA | Lahore, PAK | Dubai, UAE
(855) – 833 – 7775
(703) – 646 – 3043