What is the most expensive bug in SDLC

Inexpensive identification of previously undetected security gaps


1 White Paper Software Security Cost-effective identification of previously unrecognized security gaps Representation of the 3 procedures Threat Modeling, Static Source Code Analysis and Dynamic Analysis: Fuzzing Hartmut Pohl 1 1 Prof. Dr. Hartmut Pohl, softscheck GmbH, Cologne

2 Contents 1 No software without errors Software development cycle Threat Modeling Static Source Code Analysis Fuzzing ... 7 Exemplary fuzzing areas ... 9 Web applications ... 9 Embedded systems app fuzzing Special applications Development of proprietary fuzzers Proprietary attack strings Results The most successful Method for identifying security gaps Static Source Code Analysis Fuzzing Further reading January 2013 Inexpensive identification of previously undetected security gaps Page 2 of 14

3 Management Summary In view of the two facts: Software cannot be created without errors; If this requires a check of the software and no (successful) attack without exploiting a security gap, we offer "Security Testing as a Service" as a complete process to increase the security level of our customers' software. To identify previously unrecognized security gaps, we work successfully with the help of tools and the following 7 procedures, which are also supported by the Federal Office for Information Security (BSI): 1. Security by Design: Development of security architectures for software 2. Threat Modeling: Review of the security architecture 3. Dynamic Analysis Fuzzing: Test of the executable, compiled file, no source code required. In individual cases, we use over 50 tools from the more than 300 available worldwide: Each fuzzing tool identifies different security gaps! 4. Static Source Code Analysis: checking of implementation errors. 5. Penetration Testing: to check for known security gaps. 6. Exploratory testing and manual code auditing. 7. And finally, we also identify and analyze covert functions, undocumented, hidden functions, including on smartphones and mobile devices. In the following, the 3 successful methods of Threat Modeling (Design Phase), Static Source Code Analysis (Implementation Phase) and Dynamic Analysis: Fuzzing are intended (Verification phase) are displayed; With these procedures, all previously unpublished security gaps (zero-day vulnerabilities) are identified. The use of these 3 procedures becomes more effective and cost-effective the earlier in the software development security gaps are identified, starting in the design phase if possible. It is most expensive when the software has already been delivered (release phase) and is only then corrected - see Fig. 1. In addition, these 3 procedures can be used for many areas of application: Application software (web applications, ERM, CRM, SCM, ERP, E-business, CIM etc.) and network protocols, embedded systems (including hardware) and industrial control systems (ICS) (including proprietary systems) Manufacturing Execution Systems (MES), production control systems, SCADA (control technology and systems), PLCs up to at the field level, Cyber ​​Physical Systems (CPS), Industry 4.0, in apps and applets for smart and mobile devices, in cloud computing and also in hardware. In the Smart Grid / M2M area, we have successfully secured energy management systems - EMS and machine-to-machine communication in a Smart Meter Gateway (SMGW). The use of conventional procedures to correct errors and in particular previously unrecognized security gaps, on the other hand, is very costly - many security gaps are therefore not or only after the software has been delivered to the customer. also recognized by third parties. Fig. 2 shows the exponentially increasing effort to fix (patch) security gaps. In fact, according to some studies - including by the National Institute of Standards and Technology (NIST) - from the design phase to the release phase, the patch effort increases by a factor of 100. We know from our own studies that the effort to identify security gaps with threat modeling, static source code analysis and dynamic analysis: fuzzing can be carried out comparatively very inexpensively, even if the software is already on the market. In addition to identifying previously unrecognized security gaps, these are evaluated (rating) in order to be able to decide whether all identified security gaps should be eliminated or whether patching of some can be dispensed with (prioritization). The decisive factors for these three methods are: We recommend the use of the three methods Threat Modeling, Static Source Code Analysis and Dynamic Analysis: Fuzzing, because they complement each other successfully. Dynamic Analysis: Fuzzing does not require any source code to be successful (black box test): the software is examined during its execution. For this purpose, the software can be executed in a virtual machine or on another test system. We recommend the use of up to 50 fuzzing tools, because only such a large bundle of fuzzers will identify a sufficient number of critical security gaps. If a suitable fuzzer is not available for a specific target system, softscheck will develop special fuzzers. For special target systems, softscheck also develops specific attack strings on the basis of self-developed test data. January 2013 Inexpensive identification of previously undetected security gaps Page 3 of 14

4 Security by Design Architecture Analysis Threat Modeling Static Source Code Analysis Explorative Testing: Manual Auditing Penetration Testing Dynamic Analysis: Fuzzing Secure Design Secure Implementation Requirements Product Security Design Implementation Verification Release If security gaps are identified, sporadic operational failures and unintentional data leaks - the most common consequences of security-relevant software errors - can be counteracted, thus proactively reducing high sales losses, data protection problems and damage to reputation. We are very successful with the methods we use both tool-supported and manual auditing. In a sample of 40 projects, we identified an average of 156 security vulnerabilities: Method No. Vulnerabilities No. Tools used Architecture Analysis: Threat Modeling Static Source Code Analysis Penetration Testing (76) 4 Dynamic Analysis: Fuzzing 27> 60 Total 156> 70 The Federal Office for Information Security recommends the use of Threat Modeling, Static Source Code Analysis and Dynamic Analysis: Fuzzing. With the security testing process shown here, softscheck actually identified all previously unknown security gaps in the tested target programs. After these security checks, no other / further security gaps were recognized! January 2013 Inexpensive identification of previously undetected security gaps Page 4 of 14

5 1 No software without errors Software cannot be created without errors. This makes it necessary to check the software, with a close relationship with quality assurance. Manual checks are impractical in view of the usually large amount of code. In the world of traditional software testing, security gaps are mostly ignored in the design phase and programming manuals (often excellent) are used in the implementation phase. The only objective of tests is the specified functionalities. Non-functional tests are often neglected. The Threat Modeling, Static Source Code Analysis and Fuzzing methods can be used to detect security gaps that allow the following attacks, for example: Violation of the access rules Format string attacks SQL injections Buffer overflows. 2 Software development cycle Software is produced using methods which cannot (or cannot) completely avoid security gaps. Human errors cannot be ruled out; even if programming guidelines exist, they are not (completely) adhered to and not (completely) controlled. There are effective tools that go far beyond classic testing procedures that identify security gaps as early as the design phase (threat modeling) or at the latest in the verification phase (fuzzing and penetration testing). With threat modeling, design errors are recognized by, for example, Attack trees and data flow diagrams are evaluated. In fuzzing, in order to identify previously undetected security gaps, the input interfaces (attack surface) are attacked with inputs that are not specified, which provokes malfunction of the software. Too many security gaps are still only identified once the software has been delivered to the customer. The methods for identifying security vulnerabilities are outlined below, along with a few well-known tools. As shown, the cost of eliminating software errors depends on how soon they are discovered in the software development lifecycle (SDLC). If errors are only discovered after the release at the end customer, the costs increase depending on the investigation - by a factor of 30 to see Fig. 2. Costs 35 x Application in the Field 15 x 6 x 1 x Threat Modeling Static Analysis Fuzzing Requirements / Design Implementation Verification / Testing Release Phase Fig. 2: Costs of eliminating security gaps in the software development phases The quality of software products is often due to a lack of resources in the developing company. In addition, the market dictates very short product life cycles. Threat modeling, static source code analysis and fuzzing can be used as market-driven methods for discovering software errors. January 2013 Inexpensive identification of previously undetected security gaps Page 5 of 14

6 Studies in soft check projects show that, compared to traditional methods, only a few resources are required for this. By means of threat modeling, static source code analysis and fuzzing, software manufacturers, users and end users who adapt standard software (customizing) are able to carry out software tests more effectively and cost-effectively with the tools that are suitable for their task, thereby making the software more secure : With the help of these methods, previously undetected errors and security gaps can be identified. 3 Threat Modeling This systematic and proactive procedure supports the methodical development of a trustworthy system design or architecture in the design phase. Existing system designs and architectures can also be verified with the aim of identifying, evaluating and correcting security gaps. Threat # 1 Decrypt company data and Threat 1.1 Examine encrypted file Threat 1.2 Password / key Threat Bribery of the administrator Threat Attack database Threat Brute force attack Threat keylogger Fig. 3: Fault tree in Threat Modeling Procedure in Threat Modeling Analysis of the documentation (if available), especially the Security designs or the source code. Examination of the program flowcharts Description and prioritization of already recognized security-relevant design errors The aim is to understand the security architecture, to recognize design errors and to minimize possible points of attack (attack surface). Development of attack trees - see Fig. 3: Error trees in the Threat Modeling Report Generation: Report and recommended procedures for identified security gaps. Some threat modeling tools are Microsoft Threat Analysis & Modeling, Microsoft SDL Threat Modeling Tool and Trike. The number of identified security gaps through threat modeling is significant. In one of the authors' projects, around 100 (!) Critical security gaps that could be exploited from the Internet were identified as early as the design phase. January 2013 Inexpensive identification of previously undetected security gaps Page 6 of 14

7 Rating the Vulnerabilities critical 1 high 29 medium 26 low Number identified Vulnerabilities Fig. 4: Security gaps in standard software identified with threat modeling 4 Static Source Code Analysis This method analyzes the source code without executing it (in contrast to dynamic analysis: fuzzing). In the implementation phase, the conformity with the programming language and the programming guidelines is checked - like a parser that carries out a lexical, syntactic and semantic analysis of the program code. Some static source code analysis tools are Pixy and XDepend. 5 Fuzzing With fuzzing, the robustness of the examined software is checked with the most targeted data possible. For this purpose, input interfaces that are sent to the data are identified during fuzzing. The fuzz testing process is part of the verification phase of the SDLC - see Fig. 1. Errors should be eliminated here at the latest. Because if errors are found later, i.e. after the release - the costs increase significantly see Fig. 2.Test System Identification Code Coverage Input Interfaces Proprietary Developed Fuzzer Proprietary Developed Attack Strings Fuzzer Target System 1 Target Application 2 Encryption I / A DB Target Processor ARM, AMD, IBM, Intel, Nvidia, PLC, Power PC, Qualcomm, Sun, Snapdragon, Target OS Android, CardOS, JCOB, Nucleus, OS X, QNX, Unix, VxWorks, Windows, S7, Monitor / Debugger Monitor- Client Expert Advice: Identification, Rating Proof of Concept Exploits Report Patch, Fix Fig. 5: Fuzzing process January 2013 Inexpensive identification of previously undetected security gaps Page 7 of 14

8 Once the input interfaces of the target application have been identified and the data generated by the fuzzer has been sent to the target application, a monitoring tool monitors the target application and reports any anomalies that are detected, e.g. Program crashes, high CPU or memory load etc. to the security analyst - see Fig. 5. The security expert then carries out an analysis in which, among other things, the reproducibility, the exploitability from the Internet and the The severity of the security vulnerability (Severity) can be determined. This is the most time-consuming part of the fuzzing process. The source code of the target application is not required for fuzzing, which simplifies the involvement of external security analysts. Depending on the type, fuzzers can be used locally and remotely. The local fuzzers are e.g. the command line fuzzer counted. Network-based applications are fuzzed remotely. There are also fuzzers that fuzz web applications and browsers. A distinction can also be made between so-called dumb and smart fuzzers: Smart fuzzers independently test a target program under program control - often without any further preparation or support by the user; they often require a license. Often only smart fuzzers enable penetration into the target program and testing of the program code. Dumb fuzzers cannot recognize the structure of the target program and they only generate uncontrolled input data. Because of this lack of program control, they require considerable experience on the part of the user; however, they can often be downloaded from the Internet free of charge - see Fig. 6. Fuzzing frameworks offer the best possibility of developing fuzzers that are adapted to your own target application, similar to a construction kit. However, the development effort is relatively high, especially since ready-to-use fuzzers already exist for many applications. Fuzzing frameworks are particularly suitable for new proprietary target applications such as new network protocols. Commercial tools are usually characterized by an easy-to-use graphical user interface. The only challenge is the selection of the effective fuzzers from the more than 300 known ones. The softscheck fuzzing success depends on the correct selection depending on the respective target program. Usually the use of up to 50 fuzzers will lead to a very good result. The quality of fuzzers essentially depends on the size of the tested input space (which is infinite) and the quality of the data generated. Fuzz data generation is therefore crucial for fuzzing success. softscheck can use its own data record optimized for the target program for this purpose. Some fuzzing tools are AxMan, bestorm, Defensics, FileFuzz, FTPStress Fuzzer, Fuzz, Peach, SPIKE, SPIKEfile. Fixed costs Automatic fuzzer Manual fuzzer Variable costs / required expertise Fig. 6: Divergence of product costs Personnel expenses Another important evaluation criterion for fuzzers is test coverage (code coverage): What proportion of the program code was covered by the data fed in by the fuzzer? Selected tools are also available for this. January 2013 Inexpensive identification of previously undetected security gaps Page 8 of 14

9 Exemplary Fuzzing Areas Web Applications A specialty of software security are web applications. The dynamics of websites have increased significantly in recent years. This dynamic enables interaction with users and the provision of web-based services. For a company, this usually means a simplification of business processes, significant cost savings and competitive advantages. Even services that contain valuable company-internal data can be accessed from insecure networks (e.g. Internet) due to changed requirements for use or business processes. Nowadays, ERP systems that were only accessible from the internal company network a few years ago offer interfaces to the Internet (see SAP Business Server Pages).This trend harbors a high security risk for companies and users. Compared to conventional software, web applications have a significantly higher attack surface because a large number of processes / techniques are used (database interactions, several programming and scripting languages, web servers, etc.). In addition, the HTTP protocol offers excellent analysis options due to the open request / response communication. Due to the properties shown, a comprehensive analysis of the web application for design, implementation and configuration errors is of central importance. As a comprehensive security analysis, we recommend testing the web application using Threat Modeling, Static Source Code Analysis and Dynamic Analysis: Fuzzing, in order to ensure the highest possible level of security, equivalent to conventional software. In addition, a penetration test of the web server should be carried out. Compromising the web server also means compromising the confidentiality, integrity and authenticity of the entire web application and the underlying data. It is thus possible for attackers to view all data in the web application, to change it or even to install their own malware. The attack options shown usually cause long-term monetary damage and can seriously damage the company's image. January 2013 Inexpensive identification of previously undetected security gaps Page 9 of 14

10 Embedded Systems Embedded systems are information processing systems that are integrated into a product and that are usually not directly perceived. Examples are routers, programmable logic controllers and entertainment devices (games) etc. Security gaps in embedded systems have far-reaching consequences. Starting with routers, in the electricity meter of the smart grid up to systems for monitoring and control that are used in the electrical, gas, water and oil supply, see stuxnet. Embedded systems consist of both software and hardware. As a result of the trend reversal towards ubiquitous computing, i.e. the ubiquity of computer-aided information processing as planned in Industry 4.0, the previously self-sufficient systems are now networked with one another. In addition, complex embedded systems can have their own special operating system (embedded OS) and a web interface or they can also support high-level languages. This results in a large number of input interfaces that can be used for fuzzing. Nevertheless, the identification of the input interfaces is a challenge when fuzzing embedded systems. Not every fuzzer understands every input interface. The fuzzer can work as an external application or as an application on the test system. In any case, it stands logically between the test system and the embedded system. The embedded fuzzer acts as a bridge. Monitoring poses a further challenge. It is not always possible to install a monitor on the embedded system, there are no response telegrams to request telegrams, communication is unidirectional, etc. In such scenarios, only manual monitoring (through the Experts) possible; softscheck also develops its own fuzzers and generates specific attack strings (test data). Embedded fuzzing is ideal for analyzing errors in embedded systems. Most of the methods used today for error analysis in embedded systems only take place via error simulations. These are very time-consuming and run the risk of being incomplete. Error injections can be implemented entirely on the hardware level by means of analog fuzzers. App fuzzing The security level of apps for Android, IOS and OSX or similar operating systems for smart devices (smartphones, etc.) often does not meet user expectations; In individual cases, apps enable all data (address directory, s, SMS, MMS, callers) of the smart device to be read and recorded, as well as unrestricted access to (paid) online services - without using a PC or notebook / laptop . With Threat Modeling, Static Source Code Analysis and especially Dynamic Analysis: Fuzzing the automatic generation of targeted inputs - programming errors and security gaps can be identified. Fuzzing offers a very cost-effective way of automatically finding errors and security gaps in apps on every operating system and smartphone, which can be exploited unnoticed by attackers. The fuzzing process is equivalent to that of conventional hardware and software that runs on desktop computers, apart from certain restrictions imposed by the smartphone's hardware. If necessary, you only have to develop your own monitor solution. Special applications Development of proprietary fuzzers If a suitable fuzzer is not available for a specific target system, softscheck develops a special fuzzer. Proprietary attack strings For special target systems, softscheck also develops specific attack strings on the basis of test data developed in-house. January 2013 Inexpensive identification of previously undetected security gaps Page 10 of 14

11 6 Results With Threat Modeling and Dynamic Analysis: Fuzzing, tool-based procedures are available for identifying, in particular, security-relevant software errors. In this way, zero-day attacks, which are one of the twenty most common forms of attack, can be counteracted and application security can be increased efficiently and in line with the market. The advantages are as follows: Systematic search and also successful (!) Identification of the essential security gaps. In any software: individual software, standard software such as ERP, CRM and company-specific additions, operating systems, embedded systems, etc. No source code is required for threat modeling and fuzzing - executable files or the design are completely sufficient: black boxes. Design and implementation errors as well as security gaps can only be identified when using threat modeling and fuzzing. Only when several tools of one class are used (e.g. up to 50 fuzzers) can an optimum of previously undetected security gaps be identified. The success of fuzzing can be assessed with selected code coverage tools. softscheck can use its own data record optimized for the target program for this purpose. For software manufacturers and end users, a higher return on (security) investment can be achieved through the combination of the procedures explained and the integration into the SDLC. In addition, the quality of the software can be proactively increased, time-to-market can be reduced and costs can be saved. In addition to the cost savings, the reputation of the software manufacturer can be increased through more secure software. Threat modeling, static source code analysis and fuzzing can be used for all types of application - from protocols to individual software and web applications. To support a more needs-based selection of suitable tools, the softscheck project has developed a taxonomy that will be published separately. Assessment of security gaps Critical 5 High 6 Medium 26 Low 1 Undefined Identified security gaps Fig. 7: Security gaps in standard software identified with fuzzing The number of security gaps identified with fuzzing (mostly without the source code being available) is enormous. This is the reason for the increasing spread and popularity of the method. In one project, 5 critical, 6 high-rated and 26 medium-rated (not yet unpublished) security gaps were identified in standard software that had already been delivered to customers, which could be exploited from the Internet - see Fig. 7. This although the manufacturer also has extensive guidelines in terms of programming 'safe' software exists. January 2013 Inexpensive identification of previously undetected security gaps Page 11 of 14

12 7 The Most Successful Techniques for Identifying Security Vulnerabilities Threat Modeling Vulnerabilities are identified as early as the design phase. Systematic and tool-based procedures Security gaps can be prioritized: Exploitable from the Internet and / or little effort for the attacker Static Source Code Analysis Tool-based code reading. Fuzzing black box - no source code required - executable files will do just fine! Common processes - used by the world's largest software houses for more than 5 years - also in SMEs. January 2013 Inexpensive identification of previously undetected security gaps Page 12 of 14

13 Further reading Beyond Security (Ed.): Beyond Security (Ed.): Codenomicon (Ed.): Black Box Testing. McLean Beyond Security introduces 80/20 rule for 'smart' blackbox testing in new version of bestorm. McLean Buzz on Fuzzing. Cupertino Fox, D .: Fuzzing. DuD 30, 2006, 12, Peter, M .; Karen, S .; Romanosky, S .: Complete Guide to Complete Guide to the Common Vulnerability Scoring System Version 2.0 Gaithersburg Pohl, H .: On the technology of secret online searches. DuD 31, 9, DuD_9_07_final_.pdf Pohl, H .: Pohl, H .: Pohl, H .: Pohl, H .: Inexpensive identification of security gaps with threat modeling and fuzzing. KES Zeitschrift für Informationssicherheit, Issue 2 / ification% 20von% 20sicherheitsl% c3% bccken% pdf Fuzzel work Identification of previously unrecognized security gaps and software errors with dynamic analysis: Fuzzing. annter_sicherheitsluecken_und_software-Fehler_durch_fuzzing_kes20115.pdf Your strategy is wrong, part I and II: Security in Industrial Control Systems (ICS) a + s magazine for automats and security issues 3 and 4, Systems% 20-% 20Your% 20Strategie% 20is% 20falsch.pdf Smart Meters are sure to come. newsline The magazine of the BITKOM Academy December Rathaus, N .; Evron, G .: Open Source Fuzzing Tools. Amsterdam Doyle, F .; Fly, R .; Jenik, R .; Manor, D. Miller, C .; Naveh, Y .: Open Source Fuzzing Tools. Amsterdam 2007 Sutton, M ,; Greene, A .; Amini, P .: Fuzzing - Brute Force Vulnerability Discovery. New York Takanen, A .; Demott, J.D .; Miller, C .: Fuzzing For Software Security Testing And Quality Assurance. Norwood 2008 January 2013 Cost-effective identification of previously undetected security vulnerabilities Page 13 of 14

14 Prof. Dr. Hartmut Pohl Managing Partner soft check GmbH Cologne www. soft check.com Office: Bonner Str Sankt Augustin Tel .: +49 (2241) Mobile: +49 (172) Fax: +49 (2241) check.com