Cybersecurity

 

 

https://sm.asisonline.org/Pages/Vulnerability-Rediscovery-Occurs-At-More-Than-Twice-The-Previously-Reported-Rate.aspxVulnerability Rediscovery Occurs At More Than Twice The Previously Reported RateGP0|#91bd5d60-260d-42ec-a815-5fd358f1796d;L0|#091bd5d60-260d-42ec-a815-5fd358f1796d|Cybersecurity;GTSet|#8accba12-4830-47cd-9299-2b34a43444652017-07-21T04:00:00Zhttps://adminsm.asisonline.org/pages/megan-gates.aspx, Megan Gates<p>​Multiple researchers—working independently—uncover the same security flaws more consistently than previously believed, according to a new report from Harvard.</p><p><em></em><a href="http://www.belfercenter.org/sites/default/files/files/publication/Vulnerability%20Rediscovery.pdf" target="_blank"><em>Taking Stock: Estimating Vulnerability Rediscovery</em> </a>looked at a dataset of more than 4,300 vulnerabilities discovered between 2014 and 2016 for Android, and the Chrome and Firefox browsers. Vulnerabilities are flaws that allow cyber criminals, as well as intelligence and law enforcement agencies, to gain access to targeted systems.<br></p><p>Researchers Trey Herr, Ph.D., postdoctoral fellow with the Belfer Center’s Cyber Security Project at Harvard Kennedy School; Bruce Schneier, research fellow with the Belfer Center and adjunct lecturer in public policy at Harvard Kennedy School; and Christopher Morris, research assistant at the Harvard School of Engineering and Applied Sciences, found that rediscovery of vulnerabilities happens more than twice as often as previously reported. <br></p><p>Their findings conclude that “rediscovery happens more than twice as often as the 1 to 9 percent range previously reported,” according to the report. “For our dataset, 15 percent to 20 percent of vulnerabilities are discovered independently at least twice within a year.”<br></p><p>Based on their findings, the researchers suggested that the U.S. government rethink its process for not disclosing software vulnerabilities to companies.<br></p><p>“Underlying the choices to pay for a software vulnerability, as well as government decisions to keep some a secret, are assumptions about how often those same software flaws could be discovered by someone else, a process called rediscovery,” the researchers explained.  <br></p><p>“When combined with an estimate of the total count of vulnerabilities in use by the NSA, these rates suggest that rediscovery of vulnerabilities kept secret by the U.S. government may be the source of up to one-third of all zero-day vulnerabilities detected in use each year,” the report said. “These results indicate that the information security community needs to map the impact of rediscovery on the efficacy of bug bounty programs and policymakers should more rigorously evaluate the costs of non-disclosure of software vulnerabilities.”<br></p><p>In a post for <a href="https://lawfareblog.com/rediscovering-vulnerabilities" target="_blank">LawFare</a>, Herr explained that modern government intelligence agencies must maintain some access to software vulnerabilities. </p><p>"However, the WannaCry ransomware and NotPetya attacks have called attention to the perennial flipside of this issue--the same vulnerabilities that the U.S. government uses to conduct this targeting can also be exploited by malicious actors if they go unpatched," he wrote.</p><p>The researchers also suggested that rediscovery rates are likely higher than what their research was able to conclude because they only looked at high to critical-severity vulnerabilities.<br></p><p>For instance, records from a bug bounty company mentioned in the study “indicate that low- and medium-severity vulnerabilities are rediscovered more frequently than high- and critical severity bugs, to which this study is constrained,” the researchers wrote. “As it is, the 15 percent to 20 percent estimate is substantially higher than previously seen.”<br></p><p>The researchers plan to present the paper and discuss its findings at <a href="https://www.blackhat.com/us-17/briefings/schedule/#bug-collisions-meet-government-vulnerability-disclosure-7587" target="_blank">BlackHat USA</a> in Las Vegas next week.</p>

 

 

https://sm.asisonline.org/Pages/Trump’s-Cybersecurity-Executive-Order-Well-Received-by-Experts.aspx2017-05-12T04:00:00ZTrump’s Cybersecurity Executive Order Well Received by Experts
https://sm.asisonline.org/Pages/Seminar-Sneak-Peek---Moving-to-the-Cloud-Repositions-Security.aspx2016-08-16T04:00:00ZSeminar Sneak Peek: Moving to the Cloud Repositions Security
https://sm.asisonline.org/Pages/New-Data-Rules.aspx2016-08-01T04:00:00ZNew Data Rules

 

 

https://sm.asisonline.org/Pages/Vulnerability-Rediscovery-Occurs-At-More-Than-Twice-The-Previously-Reported-Rate.aspx2017-07-21T04:00:00ZVulnerability Rediscovery Occurs At More Than Twice The Previously Reported Rate
https://sm.asisonline.org/Pages/Report--Most-InfoSec-Professionals-Think-Their-Companies’-Security-Solutions-Are-Outdated.aspx2017-07-14T04:00:00ZReport: Most InfoSec Professionals Think Their Companies’ Security Solutions Are Outdated
https://sm.asisonline.org/Pages/Survey-Of-InfoSec-Professionals-Paints-A-Dark-Picture-Of-Cyber-Defenses.aspx2017-07-07T04:00:00ZSurvey Of InfoSec Professionals Paints A Dark Picture Of Cyber Defenses

 

 

https://sm.asisonline.org/Pages/DHS-Official-Says-Russia-Tried-to-Hack-21-States-in-2016-Election.aspx2017-06-21T04:00:00ZDHS Official Says Russia Tried to Hack 21 States in 2016 Election
https://sm.asisonline.org/Pages/Most-U.S.-Hospitals-Have-Not-Deployed-DMARC-To-Protect-Their-Email-Systems.aspx2017-06-16T04:00:00ZMost U.S. Hospitals Have Not Deployed DMARC To Protect Their Email Systems
https://sm.asisonline.org/Pages/Book-Review---Social-Media-Risk-and-Governance.aspx2016-11-01T04:00:00ZBook Review: Social Media Risk and Governance

 

 

https://sm.asisonline.org/Pages/Vulnerability-Rediscovery-Occurs-At-More-Than-Twice-The-Previously-Reported-Rate.aspx2017-07-21T04:00:00ZVulnerability Rediscovery Occurs At More Than Twice The Previously Reported Rate
https://sm.asisonline.org/Pages/Business-Theft-and-Fraud--Detection-and-Prevention.aspx2017-07-17T04:00:00ZBook Review - Business Theft and Fraud: Detection and Prevention
https://sm.asisonline.org/Pages/Survey-Of-InfoSec-Professionals-Paints-A-Dark-Picture-Of-Cyber-Defenses.aspx2017-07-07T04:00:00ZSurvey Of InfoSec Professionals Paints A Dark Picture Of Cyber Defenses

 

 

https://sm.asisonline.org/Pages/Security-Incidents-Caused-By-IoT-Devices-Could-Be-‘Catastrophic,’-Survey-Finds.aspx2017-05-31T04:00:00ZSecurity Incidents Caused By IoT Devices Could Be ‘Catastrophic,’ Survey Finds
https://sm.asisonline.org/Pages/DHS-Warns-Congress-Of-Security-Threats-to-Government-Mobile-Devices.aspx2017-05-05T04:00:00ZDHS Warns Congress Of Security Threats to Government Mobile Devices
https://sm.asisonline.org/Pages/Cyber-Travel-Tips.aspx2017-05-01T04:00:00ZCyber Travel Tips

 You May Also Like...

 

 

https://sm.asisonline.org/Pages/It-Takes-a-Network.aspxIt Takes a Network<p>​After more than four years of investigation, a global investigations team of 57 agents commenced an operation to take an international criminal infrastructure platform known as Avalanche offline at the end of November 2016. </p><p>Launched in 2009, the Avalanche network was used to facilitate malware, phishing, and spam activities. Criminals used the network to send more than 1 million emails with damaging attachments or links each week to victims in 189 different countries, according to Europol.</p><p>“The Avalanche network was used as a delivery platform to launch and manage mass global malware attacks and money mule recruiting campaigns,” a Europol press release said. “It has caused an estimated €6 million in damages in concentrated cyberattacks on online banking systems in Germany alone.”</p><p>German authorities began investigating the Avalanche network in 2012 after ransomware spread by the network infected several computer systems, and millions of private and business computer systems were injected with malware that allowed criminals using the network to obtain bank and email passwords.</p><p> “With this information, the criminals were able to perform bank transfers from the victims’ accounts,” Europol said. “The proceeds were then redirected to the criminals through a similar double fast flux infrastructure (an evasion technique used by botnets), which was specifically created to secure the proceeds of the criminal activity.”<img src="/ASIS%20SM%20Callout%20Images/0717%20Feature%204%20Stats.png" class="ms-rtePosition-2" alt="" style="margin:5px;" /></p><p>German authorities investigating the network found that Avalanche was using as many as 500,000 infected computers worldwide. After analyzing 130 terabytes of data, they were able to identify Avalanche’s server structure. Working with the U.S. Attorney’s Office for the Western District of Pennsylvania, the U.S. Department of Justice, the FBI, Europol, Eurojust, the Verden Public Prosecutor’s Office, and the Lüneburg Police arrested five individuals, conducted 37 searches, seized 39 servers, and took 221 additional servers offline via abuse notifications.</p><p>“Avalanche shows that we can only be successful in combating cybercrime when we work closely together, across sectors and across borders,” said Julian King, European Union commissioner for the Security Union, in a statement. “Cybersecurity and law enforcement authorities need to work hand-in-hand with the private sector to tackle continuously evolving criminal methods.”</p><p>International law enforcement cooperation on investigations has always been important, but it has become critical as more crimes are taking place in cyberspace—beyond national borders.</p><p>“Criminals have figured out that borders mean absolutely zero, yet for countries and law enforcement agencies, sovereignty is important—our authorities generally remain within our borders,” says Richard Downing, U.S. Department of Justice (DOJ) Criminal Division acting assistant attorney general.</p><p>And that leads to complications when victims of a crime are in one country, the offender is in another country, and evidence of the crime is in yet another country. </p><p>“Of course, nowadays, it’s more likely to be that you have victims in 20 countries, offenders in 20 countries, and the evidence in 20 other countries,” Downing adds. “Criminals understand this problem for us, and they exploit it.”</p><p>To find out how law enforcement is addressing this problem, Downing led a panel discussion with law enforcement officials at the 2017 RSA Conference in San Francisco to share how agencies are working together to combat cybercrime. </p><h4>Information Sharing</h4><p>Law enforcement agencies use various avenues to legally share information with other nations, including treaties, conventions, and investigative teams.</p><p>One type of agreement is called a Mutual Legal Assistance Treaty (MLAT), which allows law enforcement to exchange evidence and information in criminal cases and related matters. In the United States, MLATs are negotiated by the U.S. Department of State in cooperation with the DOJ to help facilitate cooperation during investigations. The United States has MLATs with the European Union, as well as with numerous other nations around the world.</p><p>These treaties are often referred to as an “18th century tool for a 21st century law enforcement,” says John Lynch, DOJ Criminal Division Computer Crime and Intellectual Property section chief. “But over the last 30 years, we’ve innovated in the sense that we’ve gone from this very slow court process to mutual legal assistance treaties.”</p><p>And building off those MLATs is the Convention on Cybercrime, which was completed in 2001 and went into effect in 2004. Sometimes referred to as the Budapest Convention on Cybercrime, it was the first international treaty that sought to address Internet and computer crime by harmonizing national laws, enhancing investigative techniques, and increasing international cooperation.</p><p>The Council of Europe drafted the original convention, but Canada, Japan, South Africa, and the United States also played a role in its creation. Since going into effect in 2004, 52 nations have ratified the convention. Russia, Brazil, and India are among the nations that have not joined.</p><p>The convention “provided innovation in that it recognized that cooperation had to occur quickly, and so it recognized an [evidence] preservation scheme,” Lynch adds. </p><p>This preservation scheme was implemented via the Group of Eight (G8)—France, Germany, Italy, the United Kingdom, Japan, the United States, Canada, and Russia—through the 24/7 Network made up of prosecutors and police officers who work to quickly preserve evidence for cybercrime investigations. </p><p>For instance, they often make requests to Internet service providers to freeze data so it can be obtained for an investigation. The government authorities then use existing MLATs to obtain the data and begin their investigation.</p><p>And as cybercrime has evolved and increased during the past decade, countries have started using joint investigative teams—what Lynch calls a hybrid of MLATs and police-to-police cooperation. </p><p>These teams “usually consist of some sort of agreement to essentially conduct an investigation together, and then establish rules of the road for how information is going to be exchanged and how it’s going to be treated by each of the departments,” he says. “Europe, in particular, has taken the lead be­cause of the need for close cooperation among those countries.”</p><p>This type of process is key for cybercrime investigations, Lynch says, because the most efficient way to tackle the threat is by running a joint investigation where police-to-police cooperation, real-time sharing, and MLATs combine to authenticate evidence as it’s recovered.</p><p>An example of this is the takedown of the Avalanche network. Steve Wilson, head of business for the European Cybercrime Centre (EC3), was involved in the investigation into Avalanche and said it worked because it used the joint investigative team method.</p><p>“We brought together large groups of investigative officers from across the world, all under one roof so they could share evidence and problems, and get things done together,” Wilson says. The EC3 brought together 57 officers—40 on day shift and 17 on night shift—as well as industry partners to help locate Avalanche’s server structure and identify those involved. </p><p>“We were dealing with probably one of the most complex cybercrime gangs we had ever seen,” Wilson says, adding that Avalanche had infiltrated 880,000 devices and 200 servers around the globe—37 of which were eventually seized by law enforcement.</p><p>Coordinating the investigation into Avalanche was a “huge challenge for us,” Wilson says, and it required using the MLATs Europe had with the DOJ and other nations to conduct the investigation, share information, and ultimately decide on how to prosecute the individuals involved.</p><p>“We arrested five key individuals who were running this network; and if any of you have an idea that cybercrime is committed by…teenagers behind computers, when we searched the house of one of the main individuals involved in this, he began shooting at the police with an AK-47,” Wilson says. “Cybercrime is now every bit as bad as serious organized crime. And investigating these international networks actually takes a network, so that’s how we’re starting to tackle this.”​</p><h4>Prioritizing Cases</h4><p>Another issue facing law enforcement investigating cybercrime is coordination among different agencies on what crimes are being investigated—so agencies aren’t stepping on each other’s toes or potentially tipping criminals off.</p><p>One way the FBI is staying abreast and informed about other investigations is by communicating regularly with Europol, and within the Bureau itself, about what cases are being worked on, says Steven Kelly, FBI International Cyber Crime Coordination Cell (IC4) unit chief.</p><p>“The best way we can help is when we’re getting investigators together, we’re getting requests for information from them, and then we’re seeing what it is that folks are asking about, we’re reporting on that, and helping enrich that feedback,” he explains. “That helps us to know what people are working on and interested in.”</p><p>The IC4 has also tried to prioritize cases to ensure that it’s focusing on the top-level schemes and actors. “Because there’s so much crime, if we take an uncoordinated approach—a country and agency are working on this, and we’re working on that—and all these investigations are taking two, three, four, or five years, we’re never going to have an impact on the crime problem,” Kelly says. </p><p>To prioritize cases, IC4 works with Europol and Interpol to develop a project plan for cases and initiatives it wants to prioritize for the next year. It then reviews and refreshes that plan every six months, most recently in April 2017.</p><p>“That’s a very useful process for getting on the same page and deciding what’s the important thing you want to focus on so we can actually focus on it and drive progress,” Kelly adds. </p><p>The FBI also depends heavily on the private sector to help inform the Bureau about what it should be investigating. </p><p>One initiative that keeps this dialogue open is the National Cyber-Forensics and Training Alliance (NCFTA) in Pittsburgh, Pennsylvania. The NCFTA is a nonprofit founded in 2002 that focuses on identifying, mitigating, and neutralizing cybercrime threats around the globe. </p><p>“The NCFTA operates by conducting real time information sharing and analysis with subject matter experts in the public, private, and academic sectors,” according to its website. “Through these partnerships, the NCFTA proactively identifies cyber threats in order to help partners take preventative measures to mitigate those threats.”</p><p>To do this, the NCFTA provides forums for partners, staff who spec­ialize in their respective initiatives, meetings and events for targeted cyber initiatives, intelligence feeds, monthly initiative calls on trends, and assessments and reports based on NCFTA intelligence.</p><p>The NCFTA is a “great platform for banks and tech firms to come together and share information, and help tip law enforcement off as to what’s important,” Kelly adds. “And if we have questions on our investigation, we can ask them.”</p><p>This model has been so effective, Kelly says, that the NCFTA is expanding its offices into two new locations: one in Newark, New Jersey, to focus on the financial sector; and one in Los Angeles, California, to focus on the technology and entertainment industries.</p><p>EC3 is also getting involved in the NCFTA after Wilson signed a memorandum of understanding with the center while at the RSA Conference in February. EC3 is making this move, Wilson says, because it mirrors similar efforts to partner with the private sector in Europe.</p><p>“We’ve got advisory groups from industry, Internet service providers, and the security industry and financial services,” he says. “We meet three times a year in relation to the problems they see…and very much in the last year we’ve recognized that law enforcement has been guilty of telling industry what they should be reporting and what they should do.”</p><p>In an effort to change that, EC3 has tried to be more open and encourage industry to bring its top two or three main problems to see how they overlap with law enforcement. “It’s really surprising how many common problems we have,” Wilson says.</p><p>Since adopting this approach, EC3 has introduced a European threat assessment that allows law enforcement to focus on the key priorities for the industry in each European country. It’s also helped foster better relationships with the private sector, which Wilson says Europol depends on for the assistance.</p><p>“We will never have staff at the top level that industry has,” Wilson explains. “We depend on that assistance, and what I’m seeing increasingly is the willingness of industry to work with us pro bono to do something—to put something good back into it.”</p><p>This dynamic is similar in the United States, according to Lynch, who says that the DOJ has found it can cooperate with the private sector to accomplish things neither law enforcement nor industry could do on its own, either due to lack of authority or expertise in an area of cyber.</p><p>“We have figured out ways so that we’re sitting together, we’re sharing information using established protocols, and can effectively take down a botnet or a criminal organization while respecting privacy and adhering to the national laws and the constitution of the United States,” Lynch says.​</p><h4>New Challenges</h4><p>While law enforcement and industry have been cooperating in some areas, a new challenge stemming from a court case involving Microsoft might prohibit future collaboration.</p><p>The case (Microsoft v. United States, U.S. Court of Appeals for the Second Circuit, No. 14-2985, 2017) was brought when Microsoft challenged a search warrant issued by a court in New York City for information that was in Microsoft’s possession but stored in a data center in Ireland.</p><p>Microsoft acknowledged that it could access the information from inside the United States, but said that because the information was stored outside of the country, the U.S. Electronic Communications Privacy Act and the U.S. Stored Communications Act did not require it to provide the information to law enforcement.</p><p>Instead, Microsoft argued, the U.S. government should use its MLAT with the Irish government to request the information.</p><p>The DOJ sued Microsoft, and a U.S. district court sided with the government. Microsoft appealed the decision, however, and the U.S. appeals court agreed with Microsoft in a ruling issued in July 2016. </p><p>The U.S. Second Circuit Court of Appeals explained that the Stored Communications Act “does not authorize courts to issue and enforce against U.S.-based service providers warrants for the seizure of customer email content that is stored exclusively on foreign servers.”</p><p>Lynch says that the DOJ is still weighing its options about whether to appeal the Second Circuit’s ruling, but in the meantime the decision will have some effect on the U.S. government’s ability to get access to information for investigations.</p><p>“On the one hand, not everyone stores their data the same way Microsoft does,” Lynch explains. “For example, Google stores its information all over the world—it sometimes splits it up and puts it into databases so it doesn’t even assemble the data until there’s a request. And in those cases, Google has made the choice that the information is only available in the United States.”</p><p>Google’s approach has also caused problems for international law enforcement wanting access to information the company has in its servers. </p><p>“Because for information located outside the United States, there’s essentially no law that can reach the data—the United States can’t reach it because of the Microsoft decision,” Lynch adds. “Foreign law enforcement can’t reach it because there’s no one in that country who has authority to access the data.”</p><p>The DOJ has also challenged Google’s position, and a district court in Philadelphia sided with the government requiring Google to turn over data to law enforcement, but the matter is far from settled.</p><p>“There’s going to be ongoing litigation in this area, and it continues to be a very difficult issue for law enforcement,” according to Lynch “We’re trying to grapple with it, because it is a problem when we can’t get the data under any regime. It can stymie an investigation altogether.”</p><p>Another major challenge for law enforcement is the perception that there are no consequences to committing cybercrime—few people appear to be charged, arrested, and then convicted of cybercrimes. This is a problem because “we’re not going to develop and build a deterrence model for cybercrime if we can’t get our hands on these people,” Kelly says. </p><p>As of February 2017, there were 123 individuals who had been charged with U.S. cybercrimes but have not been arrested, Kelly says. </p><p>“It’s a lot of people who have not been brought to justice because they are all over the world,” he explains. “They are in places we can’t get them—maybe there’s not an extradition treaty, and that’s a problem. If we’re spending a couple of years to make a case, bring it to a grand jury, get it charged, and then we can’t get the guy or gal, then that’s a problem. We’re not going to deter cybercrime if people continue to act with impunity and in safe havens.”</p><p>A recent example of this was the DOJ’s charges against two Russian spies and two criminal hackers in connection with the 2014 Yahoo data breach. One of the hackers, Karim Akehmet Tokbergenov, 22, was a Canadian national and was arrested. The other three individuals—Dmitry Aleksandrovich Dokucahaev, Igor Anatolyevich Suschin, and Alexsey Alexseyevich Belan—remain at large because Russia does not have an extradition agreement with the United States.</p><p>To address this problem, the FBI is looking at how it keeps track of cases where an individual has been charged with a cybercrime but has not been arrested. If it’s a priority apprehension, such as for a major crime, then the FBI will look at its options to possibly arrest the individuals while they are on vacation or traveling to a country that does have an extradition treaty with the United States.</p><p>And while Russia doesn’t have an extradition treaty with the United States and often refuses to extradite its own nationals, it has been known to cooperate with law enforcement for certain types of crimes, such as child exploitation charges.</p><p>“This is the one area where countries drop their individual stances,” Wilson says. “Police forces drop their egos and agree that the only thing to do is work together. I’ve seen some countries we’ve spoken about here who will not cooperate on extradition, but they will take immediate action against people who are passing out child pornography.”</p><p>Wilson says that law enforcement should use cases and moments of collaboration like this to open a dialogue about how they can work together to extradite individuals facing cybercrime charges.</p><p>“We need to keep these channels open to see if these countries will take on some of these investigations, because if we can’t have these people—if there’s no consequence to commit cybercrime—they’ll just continue to commit time and time again,” Wilson adds.</p><p>And for cases where dialogue isn’t effective, Wilson says that the European Union is looking at the possibility of using diplomatic responses and sanctions to pressure nations into cooperating. </p><p>The EU already has an agreement that if there is a terrorist attack on a member state, all of the members will stand together in response—whether it’s issuing a statement of condemnation or taking military action.</p><p>“There’s a process coming underway right now in the EU to look at the practicalities of this in relation to cyber—to actually put a consequence back to a country that either condones or actively decides to push people to commit this type of crime,” Wilson says. </p><p>The United States has taken a similar approach. Former President Barack Obama issued an executive order that allows the president to place sanctions on a nation and other actors in response to cyberattacks. </p><p>“At the end of last year, we actually implemented [the order] against a couple of actors who had been charged in the United States with ransomware schemes, botnets, and involvement in some major data breaches,” Lynch says. </p>GP0|#91bd5d60-260d-42ec-a815-5fd358f1796d;L0|#091bd5d60-260d-42ec-a815-5fd358f1796d|Cybersecurity;GTSet|#8accba12-4830-47cd-9299-2b34a4344465
https://sm.asisonline.org/Pages/Flawed-Forensics.aspxFlawed Forensics<p>​Scientifically sound forensic evidence is one of the cornerstones of the U.S. legal system. But recent research by a presidential advisory committee has questioned the soundness of some evidential techniques. This is only the latest critique of the practices of forensic science, which has faced a call for reform from some quarters.    </p><p>The most recent research has its roots in another report, which was issued in 2009 by the National Research Council on the state of the forensic sciences. That report, conducted at the behest of the U.S. Congress, was highly critical; among many other things, it found that strong protocols and standards for reporting on and analyzing evidence were lacking. </p><p>In response, various initiatives were undertaken by different U.S. government agencies, and the National Commission on Forensic Science, aimed at raising forensic standards, was formed. Additionally, in 2015, the Obama administration asked the President’s Council of Advisors on Science and Technology (PCAST) to investigate additional scientific steps that could help ensure the validity of forensic evidence used for legal matters. PCAST is a presidentially appointed advisory group of scientists and engineers.</p><p>As requested, PCAST produced a report, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, issued several months ago. </p><p>The report found two existing knowledge gaps. The first gap was the need for more clarity regarding the scientific standards upholding valid forensic methods. The second gap was the need for specific forensic methods to be evaluated, to better prove their validity.</p><p>To help close these gaps, the report examined seven forensic “feature-comparison” methods, which are used to determine whether an evidence sample is associated with a potential source sample, such as from a suspect. </p><p>The seven methods evaluated were for DNA analysis of single-source and simple-mixture samples, DNA analysis of complex-mixture samples, bite marks, latent fingerprints, firearms identifications, footwear analysis, and hair analysis. </p><p>Based on their analysis, PCAST recommended that judges should not admit into evidence four of the methods: bite marks, firearms identifications, footwear analysis, and hair analysis. </p><p>PCAST also suggested that judges be cautious when admitting DNA from complex-mixture samples, and it recommended that juries be advised that fingerprint examinations have a high error rate.</p><p>Several months after the release of the PCAST report, another significant development occurred: the U.S. Department of Justice announced that it was disbanding the National Commission on Forensic Science. Some experts now say that the absence of research and guidance from the commission could make the future task of challenging questionable scientific evidence in court even harder.</p><p>“Even if defense attorneys jump up and down and complain about [questionable evidence], they won’t have the power of a national commission to back them up,” Erin Murphy, a professor at New York University School of Law, told the Associated Press in April. “The status quo right now is to admit it all. The status quo is where things are likely to stay.”  ​ ​</p>GP0|#28ae3eb9-d865-484b-ac9f-3dfacb4ce997;L0|#028ae3eb9-d865-484b-ac9f-3dfacb4ce997|Strategic Security;GTSet|#8accba12-4830-47cd-9299-2b34a4344465
https://sm.asisonline.org/Pages/In-Search-of-Security-Metrics.aspxIn Search of Security Metrics<p>At a major insurance company headquartered in the Midwestern United States, the assistant vice president for corporate security has used an environmental risk metric for the past 12 years to help the company decide where to place office facilities around the country. The company owns or leases hundreds of facilities across the United States. Corporate security regularly collects a suite of data, assigns weights to various factors, and develops a numeric score that places each facility into a low, medium, or high category of risk. For each risk category, written policy specifies a cluster of security measures that should be in place at the site. Exceptions can be granted, but the systematic approach results in uniformity and in efficiency in decision-making and security systems contracting. Most importantly, the metrics-based approach helps senior management understand the level of risk in site selection and make informed decisions on risk management. In addition, over time, the metrics have steered the corporation toward having a smaller percentage of its locations in high-risk sites.</p><p>This example illustrates how security professionals can use metrics to determine what works, measure the value of security operations, and demonstrate security's alignment with its organization's objectives. To help security managers use metrics more effectively, the ASIS Foundation funded research to create tools for discovering, developing, assessing, improving, and presenting security metrics. By using the tools, security professionals may be better positioned to manage their operations, measure their effectiveness, and communicate with senior management. </p><p>Metrics are measurements or other objective indicators collected over time to guide decision-making. The term is sometimes used interchangeably with measurements, analytics, and performance measures. With metrics, security managers can speak to senior leaders in familiar business language, offering measurable results that correlate with investment. Without compelling metrics, security managers and their budgets rely largely on the intuition of company leadership. </p><p>Two years ago, the ASIS Foundation implemented a new structure for assessing and overseeing security research. The first test of that structure was a proposal for research on security metrics, says Linda F. Florence, Ph.D, CPP, president, ASIS Foundation Board of Trustees. "The ASIS International Defense and Intelligence Council had a special interest in the topic, having made several presentations on metrics at the ASIS Annual Seminar and Exhibits. The council formed a vision of what the security field needed, found researchers who could perform the work, and helped the researchers develop a proposal for ASIS Foundation funding."</p><p>The Foundation Research Council approved the proposal, and the Foundation sought and received funding from the ASIS Board of Directors. The result was the ASIS Foundation Metrics Research Project. The Foundation awarded a grant to Global Skills X-Change (GSX) and Ohlhausen Research to undertake the project. GSX specializes in applying validation, measurement, and standards development techniques to produce business tools. Ohlhausen Research, Inc., conducts research in security, criminal justice, and technology.</p><h4>Depth Perception<br><br></h4><p>The project's research team consisted of the author as principal investigator; subject matter expert and former Director of Information Protection for the U.S. Air Force Daniel McGarvey; Senior Analyst Megan Poore; and Technical Advisor Lance Anderson, Ph.D.</p><p>Throughout the research, which be­gan in 2013, the ASIS Defense and Intelligence Council ensured that the security practitioner's point of view was represented by serving on the project's advisory board and expert panel.</p><p>The researchers gained insights into security metrics through a systematic review of the literature, an online sur­vey of ASIS members, and lengthy fol­low- up interviews by phone. In addition, the research team was guided by an advisory board and an expert panel composed of security professionals with experience in the use of metrics. The project was completed in the spring of 2014.</p><p>The research found many books, articles, and reports discussing reasons to use metrics, characteristics of existing metrics, and methods for communicating metrics. Among the most valuable resources on security metrics were George Campbell's <em>Measures and Metrics in Corporate Security: Communicating Business Value</em> and Mary Lynn Garcia's <em>The Design and Evaluation of Physical Protection Systems</em>, as well as numerous articles in both <em>Harvard Business Review</em> and <em>MIT Sloan Management Review</em>—the latter on business metrics generally.</p><p>This noted, most sources that examine security metrics operate at a conceptual level only. The literature has few specific strategies for developing or evaluating security metrics. Likewise, descriptions of empirically sound security metrics with statistical justification and evidence are scarce. </p><p>To uncover specific uses of security metrics and to gain an understanding of the different ways in which security professionals may be using metrics, the research team invited more than 3,000 ASIS members to participate in an online survey. The survey's 20 questions asked about metrics collection, comparison to external benchmarks, return on investment, sharing and presentation of metrics, and alignment with organizational risks and objectives. The survey also examined the particulars of metrics usage among respondents.</p><p>The 297 respondents demonstrated a high degree of interest in metrics. Of the respondents who said they are not using security metrics, 78 percent said they would use metrics if they knew more about how to create and use them effectively. More than half of all respondents asked for more information from ASIS regarding metrics.</p><p>Respondents provided the research team with a detailed view of the many ways that security professionals are using metrics today, including focusing on topics, reporting data, sharing with the C-suite, aligning with organizational risk, and using a dashboard tool.</p><p><strong>Metrics topics.</strong> Respondents were asked which aspects of the security program they measure. The top five categories were security incidents, criminal incidents and investigations, cost against budget, security training and education, and guarding performance, which includes turnover and inspections. </p><p><strong>Reporting.</strong> Eighty percent of respondents who use metrics provide their metric findings to persons outside the security department. Recipi­ents of the information include senior management (79 percent of those who share metrics outside the security department), managers of other departments (59 percent), supervisors (51 percent), and people who report to the security department (47 percent). Those who share metrics provide the information quarterly (43 percent), monthly (40 percent), or annually (17 percent).</p><p><strong>Sharing.</strong> Respondents who share metrics with C-suite personnel were asked which elements they share. The top choices were security incidents (80 percent), cost against budget (62 percent), criminal incidents and investigations (57 percent), regulatory compliance (44 percent), and risk analysis process (40 percent).</p><p><strong>Alignment.</strong> Eighty percent of respondents who use metrics said that their metrics are tied to, aligned with, or part of the larger organizational risk process or organiza­tional objectives. For example, some metrics protect the company's most important product line; other metrics may support business continuity, compliance, risk management, or client satisfaction. One respondent explained that top management sets broad goals and writes plans while se­cu­rity metrics demonstrate how effective those plans are.</p><p><strong>Dashboard tool.</strong> Forty-four percent of respondents who use metrics perform their data collection, review, or sharing via a security management dashboard tool.</p><p>This research makes it possible to clearly define security's role and contribution to the organization at the tactical, organizational, and strategic levels. The report provides a working metrics tool that can help practitioners use metrics in the most effective manner. </p><h4>In the Tool Belt<br><br></h4><p>GSX and Ohlhausen Research studied the current uses of security metrics and created several resources for practition­ers. The Security Metrics Evaluation Tool (Security MET) helps security pro­fessionals develop, evaluate, and improve security metrics. A library of metric descriptions, each evaluated according to the Security MET criteria, provides valuable resources. Guidelines for using metrics can help security professionals inform and persuade senior management.</p><p>The tools, especially the Security MET, are designed to help security managers assess and refine metrics that they are using or considering, based on an intimate knowledge of conditions at their organization, in a manner guided by scientific assessment methods. </p><p><strong>Security MET.</strong> The Security MET is meant to aid and empower the security manager, not to dictate any particular security decision. By providing a standard for scientific measurement, it offers guidance for improving the inputs that go into the security professional's own decision-making process.</p><p>The Security MET is a written instrument that security managers can use to assess the quality of specific security metrics. Users can determine whether an existing or proposed metric possesses scientific validity, organizational rele­vance (such as clear alignment with corporate risks or goals), return on investment, and practicality.</p><p>The tool was developed through a comprehensive, iterative process that involved synthesizing scientific literature, reviewing security industry standards, and obtaining input from metrics experts on the project's advisory board and expert panel. Many of the criteria come from the field of psychometrics, which is concerned with the measurement of mental traits, abilities, and processes. The psychometric literature addresses the measurement of complex human behaviors, including sources of error inherent in social and organizational situations. In addition, through its connection with legal guidelines and case law, psychometric theory provides ways to address complicated legal issues related to fairness and human error.</p><p>The tool presents nine criteria for evaluating a security metric. The criteria fall into three groups: technical, operational, and strategic.</p><p><em>Technical.</em> The technical criteria include reliability, validity, and generaliz­ability. Reliability means the degree to which the metric yields consistent scores that are unaffected by sources of measurement error. Validity refers to the degree to which evidence based on theory or quantitative research supports drawing conclusions from the metric. Generalizability means the degree to which conclusions drawn from the metric are consistent and applicable across different settings, organizations, timeframes, or circumstances.</p><p><em>Operational.</em> Operational criteria include the monetary and nonmonetary costs associated with metric development and administration, as well as timeliness and the extent to which metric data can be manipulated, coached, guessed, or faked by staff.</p><p><em>Strategic.</em> Strategic criteria include return on investment, organizational relevance, and communication. Return on investment is the extent to which a metric can be used to demonstrate cost savings or loss prevention in relation to relevant security spending. Organizational relevance is the extent to which the metric is linked to organizational risk management or a strategic mission, objective, goal, asset, threat, or vulnerability relevant to the organization—in other words, linked to the factors that matter the most to senior management. Communication refers to the extent to which the metric, metric results, and metric value can be communicated easily, succinctly, and quickly to key stakeholders, especially senior management.</p><p>A score sheet is presented at the end of the Security MET. The instrument is easy to score and imposes little to no time burden on staff. Lower scores on particular criteria show where a metric has room for improvement. </p><p>Here's an example of how the Security MET can be used to evaluate a real-life metric. At a major financial services firm, employees were being robbed of their mobile phones on the sidewalks all around the office as they came to work, when they went outside for lunch, or when they left to go home. The firm identified hot spots and times for phone theft and applied extra security measures. After reaching a maximum of 40 thefts in a two-month period, the number soon declined to zero.</p><p>Evaluating the metric with the Security MET provides some valuable insights. The metric—the number of mobile phone thefts—is highly reliable, as it is based on incident reports from employee victims, police reports, and video surveillance. Its validity appears to be confirmed by the outcome—that problem was eliminated. Collecting the data has little marginal cost, as the company already tracks and trends security incidents. Its organizational relevance is high, as it aligns with the firm's goal of attracting workers to the central business district. As for communication, it is a straightforward metric that is easy to explain. In terms of return on investment, it is hard to quantify the value of keeping employees safe and continuing to attract new employees.</p><p>Thus, while the metric appears to present a reasonable return on investment, the Security MET helps the user see that developing clear proof of ROI would be one way to strengthen this particular metric. The addition of a short survey asking employees if they feel more se­cure and would recommend the company to others would provide validation for both the solution and the metric.</p><p><strong>Metrics library.</strong> The researchers de­veloped 16 summaries of metrics currently in use in the security field. The summaries were developed primarily through telephone interviews with on­line survey respondents. The summaries may serve as examples for security pro­fessionals who are considering ways to use metrics. (See box on page 58 for a complete list of topics.)</p><p>The library presents a three- to four-page summary of each metric. In addition, each metric is evaluated by several metrics experts, using the Security MET. The metrics library is presented in the full project report.</p><p>These real-world metrics come from a variety of industries including defense/aerospace, energy/oil, finance, government, insurance, manufacturing, pharmaceuticals, real estate management, retail, security services, shipping/logistics, and telecommunications.</p><p>Some of the metrics are more sophisticated and detailed than others, providing a range of examples for potential users to consider. The metrics are not presented as models of perfection. Rather, they are authentic examples that security professionals can follow, refine, or otherwise adapt when developing their own metrics.</p><p><strong>Guidelines.</strong> A key task in this research was to develop guidelines for effectively using security metrics to persuade senior management. What would make those presentations more compelling? Several recommendations emerged.</p><p>Present metrics that are aligned with the organization's objectives or risks or that measure the specific issues in which management is most interested. One of the most important measures is return on investment (ROI).</p><p>Present metrics that meet measurement standards. A metric may be more persuasive to senior management if it has been properly designed from a scientific point of view and has been evaluated against a testing tool, such as the Security MET, or established measurement and statistical criteria.</p><p>Tell a story. If the metric is prevention-focused, a security professional can make the metric compelling by naming the business resources threatened, stat­ing the value of those resources, and describing the consequences if the event occurs. Another part of a compelling story is the unfolding of events over time. Metrics can show progress toward a specific strategic goal. </p><p>Use graphics and keep presentations short. Senior managers may be interested in only a few key measures. While security professionals may choose to monitor many metrics via a dashboard interface, they should create a simpler dashboard for senior management. Some security professionals said they limit their presentations to five minutes.</p><p>Present metric data regularly. As data ages it becomes more historical, less actionable, and thus potentially less valuable. The research does not suggest an optimal interval for sharing security metrics with senior management, but the survey shows that 83 percent of security professionals who share metrics outside the department do so at least quarterly. </p><p>Future steps for helping security professionals improve their use of metrics include a webinar sponsored by the ASIS Defense and Intelligence Council and the further development of the metrics library. Other ideas under consideration include metrics training for security practitioners, the development of a tool for creating a metric from scratch and implementing it in an organization, and the creation of a library of audited— not merely self-reported—metrics. </p><p>The best security practice is evi­dence-based; without research, practitioners must rely on anecdotal information to make decisions. The ASIS Foundation continues to seek ideas for research projects that would increase security knowledge and help security professionals perform their work more effectively. </p><p>The complete project report, <em>Persuading Senior Management with Effective, Evaluated Security Metrics</em>, is available as a free download. The 196-page report contains the full text of the Security MET, the library of metric summaries (with evaluations), guidelines for presenting metrics to senior management, the project's literature review, and detailed results of the online survey.</p><p>Florence says, "We are proud to brand this quality research with the ASIS Foundation logo and share the findings with our members and the security profession as a whole. This research will help propel security from an industry to a profession, where we belong."  <br></p><p>Peter E. Ohlhausen is president of Ohlhausen Research, Inc., and served as principal investigator for the ASIS Foundation Metrics Research Project. He is a member of ASIS.</p>GP0|#28ae3eb9-d865-484b-ac9f-3dfacb4ce997;L0|#028ae3eb9-d865-484b-ac9f-3dfacb4ce997|Strategic Security;GTSet|#8accba12-4830-47cd-9299-2b34a4344465