Active content white paper

Measuring upEvaluating the return on investment (ROI) of spam filtering This white paper provides a framework for evaluating the ROI of a spam-filteringsolution, looking at both the financial benefits of stopping spam – increasedproductivity and reduced costs – and the cost considerations for implementinga filtering solution.
It is difficult to recall what life was like before email – when information flowed at asnail’s pace and an organization’s shipping and delivery accounts might rival thenational debt. Today, email has emerged as a powerful business-critical application,enabling fast and efficient collaboration between co-workers, customers, suppliers,and partners. According to the Gartner Group, North American workers spend anaverage of 49 minutes managing email each day, and that figure is expected to rise.
Furthermore, Ferris Research estimates that in 2003, management-level personnelwill work with email for an astonishing four hours every day.
However, productive use of enterprise email is threatened by the growth of spam, or unsolicited commercial email (UCE). Gartner estimates that spam represents more than 30% of email traffic within enterprises today, and that if defensive action isn’ttaken now, spam will represent more than 50% of message traffic by 2004. As theamount of spam reaching employees’ inboxes increases, the resulting cost toenterprises grows. Ferris estimates that the cost of spam to US organizations was$8.9 billion in 2002, accounting for lost productivity, consumption of IT resources,and helpdesk time. And those costs will only rise as spam continues to proliferate.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
Spam results in numerous indirect costs for an organization, including productivityloss, helpdesk complaints, and infrastructure resource consumption.
The negative consequences of spam begin with users who must spend valuable time differentiating legitimate messages from spam messages. Below is a conservative spam represent substantial lossesof productivity.
example that estimates the cost to an organization of lost end-user productivity: Average time spent reviewing a spam message Lost wages/productivity per employee per day Daily cost for a 10,000-person organization Annual cost for a 10,000-person organization*
$495,000
Productivity can be further diminished if a user decides to try to act upon, reply to,or unsubscribe from spam messages. Ironically, these actions can also increase theamount of future spam the user will receive. In addition to time spent reviewing anddeleting spam messages, costs can result when users occasionally delete a legitimatemessage (a possibility that becomes even more likely as the proportion of spam tolegitimate email increases). Accidental deletion can cause lost productivity when auser tries to locate a lost message, or if that user misses the message entirely.
Complaints to the help deskabout spam results in costs Spam is also a major source of helpdesk complaints for organizations. Some attributable to both the user andthe help desk.
complaints may be about specific messages – “I received this Viagra email that Idon’t want.” Other complaints may be about the overall volume of spam – “Can’tyou stop all of this spam in my company email account?” These complaints can betraced to a user cost and a helpdesk cost. Below is an example of the cost oforganizational complaints, assuming that one in every two employees will registerone complaint per year.
Number of complaints per employee per year Helpdesk time tracking and addressing complaint Daily cost for a 10,000-person organization Annual cost for a 10,000-person organization
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
MEASURING UP: EVALUATING THE ROI OF SPAM FILTERING Unfiltered spam burdens an organization’s messaging infrastructure well beyond the requirements of legitimate email traffic. Managing spam message traffic requires message-handling capacity, andcan expose it to the risk of legal additional network resources, mail server processing cycles, and storage capacity. If spam represents 30% of all incoming messages, and incoming messages representhalf of all message processing, organizations have no choice but to invest anadditional 15% for network and human resources to handle that email traffic. Thisultimately translates to a higher cost per user or per legitimate email processed.
These are not the only negative costs associated with spam, or even the mostpotentially destructive. Organizations face the risk of employee legal action due tooffensive content or language that can result in an unacceptable workplaceenvironment, and these organizations may feel forced to purchase insurance tosafeguard against this risk. For example, in 1996, the IT operating company atChevron spent $125 million to settle a lawsuit by 777 female employees based on adegrading email that was circulated internally. In addition, irrelevant andinappropriate spam can be harmful to overall employee morale and satisfaction.
Many organizations are now beginning to realize their obligations to filter email inorder to ensure a safe work environment.
ROI considerations when evaluating a spam-filtering solution Research has shown that spam is a critical concern among organizations. In a surveyof IT professionals conducted at the 2002 Microsoft Exchange Conference, almost75% of those surveyed regarded spam as a moderate or severe problem for theirorganization, and 60% were evaluating anti-spam solutions at that time.
Because spam can cost organizations hundreds of thousands of dollars or more eachyear in lost productivity, helpdesk complaints, and infrastructure resourceconsumption, organizations are increasingly seeking out spam-filtering solutions.
This section describes characteristics that affect the ROI of a spam-filtering solution,including spam detection accuracy and administrative costs.
The accuracy of a spam-filtering solution is measured using two primary metrics – the catch rate and the false positive rate. The catch rate refers to the percentage of optimizing the percentage ofspam detected and the number spam that is detected. The false positive rate refers to the percentage of legitimate email that is incorrectly identified as spam. Unfortunately, perfect performance onboth metrics is not possible – 100% of spam can be caught with a very high falsepositive rate, or 50-60% of spam can be caught with few or no false positives.
Both of these metrics strongly affect the ROI of a spam-filtering solution. The catchrate determines how muchof the productivity losses, and other costs associated withspam, are salvaged. For example, a solution that catches only 60% of spam messagesstill leaves the organization with 40% of the costs of spam, while a solution thatcatches 95% of spam leaves the organization with only 5% of those spam costs. Atthe same time, false positives create costs to organizations when users miss legitimateemails and must initiate helpdesk complaints or inquiries regarding the missed email.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
Organizational requirements for spam filtering vary, and an easy and cost-effectivesolution that allows an organization to meet its specific requirements will increase ROI.
Organizations have different preferences for empowering end users with regard to aspam-filtering solution. Options for organizations include: Placing spam in a quarantine to which administrators can refer if a userinquires about a specific message Placing spam in a quarantine and publishing a digest to users so they canquickly scan messages identified as spam Flagging messages as spam and delivering them to the end user, enabling theend user to choose whether to delete, route to a separate folder, etc.
Flexibility to choose easily between a variety of implementation options willoptimize user and administrator time, minimize helpdesk questions andcomplaints, and improve user acceptance, leading to a higher ROI.
Optimizing the accuracy of spam detection (maximizing catch rates and minimizing false positives) requires a high-quality set of spam-detecting heuristics, tailored to the organization, and bythe effective use of whitelisting.
which can be further enhanced by organizational fine-tuning or customization.
Organizations such as insurance or mortgage companies are examples of the needfor tuning spam detection to a specific industry or company. These companies mayreceive a high percentage of messages containing large dollar amounts, which isoften considered a characteristic of spam, so they may want to de-emphasize theimportance of this characteristic in determining which messages are likely to bespam. Initial accommodation of such company-specific characteristics will helpoptimize filtering accuracy.
Also, initial whitelisting of major business partners and common newslettersubscriptions can further minimize false positives. Depending on the trade-offsaffecting ROI, most organizations will choose to customize their spam-filteringsolution during the burn-in period to optimize accuracy.
Organizational environments have varying scalability and architecturalrequirements. A solution that can easily be deployed in a variety of environmentswill require fewer IT implementation resources, which also increases ROI.
Regardless of the environment, a message-filtering solution should allow forefficient testing, piloting, and deployment as required by the organization, evenwhen that organization grows or changes.
ROI is also affected by post-rollout administrative costs, such as maintaining spam-catching filters, managing user requests and quarantines, and reporting on activityor effectiveness.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
MEASURING UP: EVALUATING THE ROI OF SPAM FILTERING Spam techniques and characteristics evolve over time. To eliminate or minimize theamount of time an administrator spends analyzing spam and developing filters, asolution should initially provide an effective set of filters, as well as ongoing updatesthat address changing spam tactics and characteristics. Updates should be easy toapply, either manually or automatically.
Management of quarantines and end-user inquiries Another area of potential administrative costs involves managing quarantines and/orresponding to end-user inquiries regarding false positives or missed legitimateemails. A solution should enable administrators to minimize the amount of timespent on these activities, while maximizing their performance.
Administrators often require the ability to monitor and report on activity andperformance as needed, and a solution should minimize the time required togenerate required reporting.
Straightforward scaling and licensing As message volumes and user populations grow or change, administrators should beable to understand licensing requirements in order to easily shift or add neededcapacity.
Ability to address non-spam message filtering requirements Although this white paper is focused on ROI considerations for addressing spamspecifically, overall ROI may also be affected by other applications of the chosenmessage-filtering solution. A solution may also satisfy other requirements such asvirus detection and email policy enforcement (e.g. archiving, adding disclaimers tomessages, and filtering outbound mail for inappropriate or proprietary content).
Sophos PureMessage and spam-filtering ROI Sophos PureMessage provides consolidated email protection, including accurate,flexible, and easy-to-administer spam filtering, as well as virus detection and emailpolicy enforcement.
PureMessage provides excellent out-of-the-box spam filtering accuracy, featuring an approximate 80% spam catch rate, with the ability to perform customization during over 95%. False positives can befewer than 1 in 100,000.
the burn-in period to further optimize effectiveness. After initial fine-tuning, thecatch rate averages over 95%, and false positives can be minimized to fewer than 1 in100,000. This accuracy enables organizations to eliminate the vast majority of spamcosts while introducing only minimal new costs due to false positives, therebyincreasing ROI.
PureMessage attains high accuracy by using a proactive, heuristics-based approachto spam filtering. PureMessage incorporates approximately 1000 different rules andchecks, which are combined to produce an overall probability that a message is spam.
For more information on the techniques PureMessage uses to detect spam, refer tothe Sophos white paper “Safeguarding email productivity”.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
PureMessage offers easy and flexible implementation options through a web-basedadministrative interface, enabling organizations to maximize ROI based on theirdistinct situation and requirements.
In order to minimize user and administrator time spent on spam, includinghelpdesk inquiries and complaints, PureMessage allows administrators to choosefrom a variety of user implementation options. Administrators can define one ormore spam probability ranges, then choose to delete, quarantine, publish digests,and/or flag and deliver spam messages to users based on those ranges.
PureMessage allows administrators to choose the desired level of spam-filteringaccuracy and customization. PureMessage provides excellent accuracy out of thebox, and it also allows organizations to perform initial, company-specific fine-tuning to further optimize accuracy and minimize spam costs.
PureMessage provides a variety of options to support administrators during testingand rollout. Deployable on either the same hardware as the mail transfer agent(MTA) or on separate hardware, PureMessage can be configured in a variety of waysto fit an organization’s unique messaging architecture. Additionally, administratorscan test in a separate environment to predict filtering impact in production withoutaffecting email traffic, then selectively implement for all or part of an organization.
PureMessage minimizes ongoing administrative costs, and provides an easy-to-use,web-based interface for administrative activities such as reporting.
A team of spam analysts at Sophos continuously monitors spam trends andpublishes frequent updates in order to maintain and improve spam detectionaccuracy. An administrator can choose to have these updates applied manually orautomatically, with little or no impact to message processing.
Management of quarantines and end-user inquiries Administrators have several options for managing quarantines and end-userinquiries. Administrator review of quarantines can be performed centrally ordistributed among multiple administrators or levels. Alternatively, PureMessage canbe configured so that quarantine digests are automatically generated and sent to endusers, who can easily retrieve and review any quarantined messages at their leisure.
The web-based interface for PureMessage allows administrators to monitorprocesses and resource utilization, as well as perform summary reporting onmessage-filtering activity. If desired, detailed logs can be exported to in-housereporting tools for further analysis.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
MEASURING UP: EVALUATING THE ROI OF SPAM FILTERING Ability to address non-spam message filtering requirements PureMessage is a consolidated email protection solution, providing spam filtering aswell as virus detection and policy enforcement. PureMessage is based on a flexiblepolicy engine that can be used for message archiving, message disclaimers, keywordfiltering, outbound content/IP filtering, and other needs as evolving message-filtering requirements and threats occur.
PureMessage also includes an annual support package, reducing administrative costsand providing mail administrators with personalized assistance and information.
Expert support, priority response times, and rapid escalation of issues ensureoptimized performance and security.
2003 Sophos Plc. No part of this publication may be reproduced, stored in a retrieval system, or transmitted by any form or by any means without the prior written permission of the publishers.
MEASURING UP: EVALUATING THE ROI OF SPAM FILTERING Email has become an undeniable necessity for conducting business today, and organizations are looking for ways to manage the negative impact of email – spam – the cost impact of the variouscharacteristics of spam-filtering while taking advantage of all its positive impacts. The significant costs associated with spam are prompting organizations to evaluate spam-filtering solutionsaggressively. To ensure the highest ROI possible from a project to address spam,managers should evaluate the cost of spam to their organizations, then compare it tothe cost impact of various characteristics of spam-filtering solutions. By choosing anaccurate, flexible-to-implement, and easy-to-administer solution such as SophosPureMessage, organizations can maximize ROI, reduce costs and liability, andincrease employee productivity and morale.
Sophos, Inc. All registered trademarks and copyrights are understood and recognized by Sophos.

Source: http://netsense.info/downloads/spam_ROI.pdf