J. Cent. South Univ. Technol. (2009) 16: 0258-0264
DOI: 10.1007/s11771-009-0044-0
A fault injection model-oriented testing strategy for component security
CHEN Jin-fu(陈锦富)1, LU Yan-sheng(卢炎生)1, ZHANG Wei(张 卫)2, XIE Xiao-dong(谢晓东)1
(1. College of Computer Science and Technology, Huazhong University of Science and Technology,
Wuhan 430074, China;
2. Department of Computer Science, Iowa State University, Ames 50011, USA)
Abstract: A fault injection model-oriented testing strategy was proposed for detecting component vulnerabilities. A fault injection model was defined, and the faults were injected into the tested component based on the fault injection model to trigger security exceptions. The testing process could be recorded by the monitoring mechanism of the strategy, and the monitoring information was written into the security log. The component vulnerabilities could be detected by the detecting algorithm through analyzing the security log. Lastly, some experiments were done in an integration testing platform to verify the applicability of the strategy. The experimental results show that the strategy is effective and operable. The detecting rate is more than 90% for vulnerability components.
Key words: component testing; component security; fault injection model; testing strategy; detecting algorithm
1 Introduction
Component-based software engineering (CBSE) has been the research focus in the field of software engineering at present. All kinds of new component development technologies are aimed to enhance the efficiency of component development and performance. However, problems with the component reliability and security have not been solved yet. Testing the component and component system is an important process that guarantees and enhances system correctness, reliability and security. Current component testing approaches are focused on component functionality testing, which, to some extent, ensures the correctness and integrity of component functionality [1]. To the best of our knowledge, component security testing is rarely researched as a special subject, and there are not feasible approaches or technologies in detecting component security vulnerabilities. Presently, there have been few testing approaches on security testing, which are mainly derived from traditional software testing approaches [1]. However, software security testing technologies themselves have not been mature yet [2-4], since most components source codes are unavailable, which challenges the security testing of components, especially third-party components.
The research on component security testing mainly focuses on component security characterization and assessment approaches, formalization analysis approaches, perspective of dynamic component deployment testing and wrapper testing approaches. HAN and ZHENG [5] proposed the specifications for component security that could enhance the testability of component security to some extent. The specifications include resource allocation, environment configuration and method invocation. GUO et al [6] discussed testing and assessment on software security through virtual machine technology, which was not presented in details. NISSANKE [7], BRYANT et al [8], ZHONG and EDWARDS [9] advanced formalization and access control approaches on testing and analyzing component security. The formalization approach can enhance the testing accuracy of component security, but it has such shortcomings as small application scope, difficult operability and automation that also exist in the access control approach. BERTOLINO and POLINI [10] proposed a framework of component deployment testing, in which a spy class was added in the tested component to collect and compare the resources allocation and running state of the tested component. HADDOX et al [11] advanced a wrapper testing approach for the third-party components. This approach adds input and output testing interfaces to the tested component, which allows the data to flow into and out of the component at the public wrapping interface level. Thus the third-party component will be better understood and tested. CHEN et al [12] and LU et al [13] proposed testing approach and testing model of component security based on dynamic monitoring. The approaches discuss the detection of component vulnerabilities from the monitoring. However, the approaches lack feasible fault injection strategy and effective mechanisms, which may cause security vulnerability of component.
The vulnerability of component, especially that of the third-party component, is closely related to specific system environment [14-15]. In view of security strategy, it is necessary to consider the following environment entities: environment variable names, directory of running procedure, temporary files, memory space, registration information, storage files, processes, external dynamic link library (DLL) resources, and network. As for the attributes of most components, fault injection technology is suitable for testing their security very well. In this work, the relationship between the fault injection and component security testing was addressed. A fault injection model was given for testing the component security. Based on the fault injection model, a new testing strategy and detection algorithm were presented for detecting component vulnerability.
2 Fault injection model
As a testing technology, fault injection technology is that faults are purposely generated according to the specific fault model, and then imposed on the tested system to accelerate the occurrence of system’s errors and failures [16-17]. The tester observes the feedback information after faults are injected into the system. Through analyzing testing information, we can validate and evaluate the tested system. DU and MATHUR [15] proposed that a system was composed of an application and its environment. The codes that do not belong to the application belong to the environment. Thus, fault injection can be divided into application fault injection and environment fault injection. The environment fault injection is an automatic software testing approach. This approach has been widely used in some areas such as testing security protocols [18-19]. A feasible fault injection system should conduct a comprehensive vulnerability testing for the tested software, and try to inject every possible fault into the tested software so as to achieve a testing software defect purpose [14]. Software fault injection testing technique including mutation testing focuses on software program with source codes at present, which, to some extent, ensures the correctness of software program. To the best of our knowledge, software fault injection testing is rarely used in testing third-party software of which source codes are unavailable. There are no feasible fault injection approaches in testing third-party component vulnerabilities. A fault injection model that aims at testing third-party component security was presented in this work.
Application users include visual user (human being) and invisible users such as application programming interface (API), operating system (OS), and file system [20]. We adopted the fault injection model (FIM) shown in Fig.1 in the strategy based on the theory about invisible users of software. FIM is a six-tuple {IP, M, DF, PRS, NET, REG}, and IP (interface parameter), M (memory), DF (disk file), PRS (process), NET (network), and REG (registration) mean interface parameter, memory, disk file system, external component or process, network, and registration information, respectively. FIM shows the above-mentioned six aspects injected into the tested component. According to different test objects and requirements in practical application, FIM can be extended. According to the characteristics of component object model (COM) component, this strategy adopts the combined injection approach composed of the fault injection of static interface parameter and dynamic environment, so as to test COM security from various aspects. Interface analysis can attain the necessary testing information such as component objects, methods, interfaces and interface parameters, and then faults can be injected into component methods. Environment fault injection refers to injecting faults into interactive level between the operating system and component when the tested component runs. Its injection framework is shown in Fig.2.
Fig.1 Fault injection model for component security
Fig.2 Framework of environment fault injection
At the interface parameter level, testing approach firstly analyzes the information of component interface such as interface name, interface functions and function parameters, and then faults such as input parameters, parameter types, and parameter sequence are injected into the tested component. Memory faults are mainly classified into the following four categories: restriction of memory for tested component, lower memory, access control for memory and invalid address of memory. These faults are simulated by testing mechanism and then injected into the tested component. File system faults mainly include the restriction of reading and writing files, invalid files, corrupt directory and write-protect violation, etc. These faults are simulated in the same way, and then injected into the component for testing. The process faults mainly include resource insufficiency, invocation components incapable of finding and corrupt DLL file, etc. The faults are deliberately injected into the process invocation for triggering security exceptions. In the network level, faults are mainly injected into network components and network connection. These simulated faults include disconnection, service denied, no ports available and network congestion, etc. Registry faults such as corrupt registry, modification denied, error authentication and encryption, are injected into register table and tested component.
An example component named AcroPDF.dll is presented for describing the FIM. AcroPDF is an ActiveX control in Adobe Reader and Acrobat. Component version number is 7.0.8.0. The fault injection model for AcroPDF component is shown in Table 1.
Table 1 Fault injection model for AcroPDF component
For a given tested component, the fault injection algorithm is given as follows based on the above fault injection model.
(1) Inject parameter faults into the tested component if the component interfaces have input parameters, otherwise go to Step (2). At the same time, the testing results are recorded.
(2) Inject the memory faults into the running component, and record the injecting results.
(3) Inject the file faults, process faults, register information and network faults into the tested component, and record the injection results.
(4) Inject the combination faults of the smallest two factors coverage into the tested component, and record the testing results.
(5) Inject the combination faults of the smallest three factors coverage into tested components, and record the testing results.
(6) Summarize and analyze the recorded testing results, and write the analyzing summary into the testing report.
3 Testing strategy
Based on the fault injection mechanism that is an important technology usually used in the software robustness testing and software error locating, a testing strategy of component security named fault injection model-oriented component vulnerability detecting (FCVD) is proposed.
3.1 Framework of testing strategy
Based on the fault injection model described above, the FCVD mainly contains the following functions.
(1) Analyze the interface information of COM component, and attain necessary type information.
(2) Automatically generate testing procedures of component method-level, and further generate test cases of interface methods to achieve faults injection testing for interface parameters.
(3) Compile test fixtures and generate test driver of component.
(4) Inject the environment faults into the driven component for security testing.
(5) Dynamically monitor the running process of the tested component, record the exceptions of component security, and then write them into security log.
(6) Assess and analyze the component security. The framework of the proposed strategy is shown in Fig.3.
Fig.3 Framework of FCVD
3.1.1 Interface analysis
For the source codes of third-party COM component are unavailable, the type information of component must be attained through special approach before testing. This strategy can get the specific component information such as component methods and interfaces through analyzing the type information. Type library includes the interface information described by binary file. The type-library includes all information that is basically the same as interface definition language (IDL) files. The type library can be a separate binary file. It can also be the resources of dynamic link library or executable file. The following information can be attained by analyzing the type library.
(1) The name and version of type library.
(2) The number of type information.
(3) The interface type (e.g. CoClass, Interface and Dispatch), function number and variable number.
(4) Function name, type of return value, flag, invocation type (e.g. Function, Get, Put), parameter number, optional parameter number, parameter name, parameter type and parameter direction.
In addition, all the above-mentioned information is stored in the pre-defined XML (eXtensible Markup Language) format file after analyzing.
3.1.2 Generation of test file
The test file, which includes simple test cases, is generated by the module according to the type information stored in XML file. The main implementing steps are as follows.
(1) Read the XML file using Xpath technology, and get interface information.
(2) Generate test file using CodeDom technology according to interface information.
(3) Attain the initial parameter values of interface methods according to security requirement specification.
3.1.3 Compiling and driving
Auto-compiling module mainly automatically compiles the tested COM component, test case file [21-22] and related procedure set (dll format) into dynamic link library file (dll format) that is called by the driver. Implemental main functions are as follows: reading configuration file (config.xml) to get the COM component of designated location and COM TLB (Type LiBrary) file, completing component registration and choosing test case file and related procedures set (dll format) to compile them into dll file that is called by the driver.
3.1.4 Environment fault injection
Environment faults are injected into the running component from memory, disk system, process, network, registration information and other relevant environment. According to the type of injected faults, injection module intercepts system API that is interactive between operating system and running component in the process. The implemental process is to replace system dll file with pre-defined pseudo-dll file. According to the testing requirements of fault injection, some original system dll files are replaced with some pseudo-dll files that are created in advance with the same name. As some API functions do not need to be intercepted, testing system can directly pass the parameters to the corresponding system API functions for processing and returning the results. As others need to be intercepted, testing system can add interception codes in the pseudo-dll files, and at the same time, the faults can be injected.
3.1.5 Dynamic monitoring
Dynamic monitoring module mainly uses the debugging API technology of Windows to control the test driver, which monitors the operation state of component. The main implemental functions are as follows: monitoring the stack values of run-time components (function input parameters), the stack values at the end of running function (function output parameters), and CPU (central processing unit) register values (function return values). The basic principles are as follows: setting breakpoint at the entrance of function after getting the entrance address of function, triggering the debugging interrupt operation when procedures call breakpoint function, then suspending the running of goal procedures for reading the memory of goal procedures and attaining the available memory information of goal procedures in the end. By analyzing the memory information and abnormal log, component security vulnerabilities can be detected.
3.1.6 Security analysis
The main functions of security analysis are to analyze the component security according to the security requirement specification and dynamic monitoring log, and to generate a testing report. Two cases of potential security vulnerability are as follows: (1) there exist exception codes and interface method in exception information, and (2) there is not exception information, but the running of internal component violates the restrictive conditions of security requirement.
3.2 Requirement specification of component security
Requirement specification of component security is described by XML according to certain schema. Requirement specification is provided by developers, or obtained through analyzing interface information, function specification and IDL information. HAN and ZHENG [5], JABEEN and JAFFAR-UR-REHMAN [23] proposed related requirement specification for component testing, which is not suitable for component security testing. We improved the requirement specification for component security testing. The key elements of security requirement specification mainly include component configuration, component interface and security requirement specification. The format is shown as follows.
The item component-configuration describes component configuration information including develop- ment incorporation, developer name, development date and component version, etc. The information facilitates component usage and component testing. Component-interface is a very important item in the requirement specification. It describes component structure information such as component methods, component attributes, component inheritances and component invariants, etc. The format is shown as follows.
Interface-information item describes the interface information such as interface number, interface name, etc. Inheritance-hierarchy item describes the inherited attributes of class. Attributes item describes the attributes and variant values of object. Invariant item describes component transition of states and the constraints of invariant values. Method item describes the execution profiles of component methods. It includes some information such as pre-asserts, pre-conditions, method parameters, parameter type, parameter conditions, post-asserts and post-conditions, etc.
The item security requirement-specification describes some information such as component interface, development environment, running environment, memory usage and constraint conditions for method resources that are related to component security. The format is shown as follows.
Interface item describes security constraints in the interface such as the constraints of interface function invocation, etc. Environment item describes the security requirements of configuration environment, which are required by the tested component. Memory item describes the security requirement of memory usage for the tested component. Method resource constraint is a very important item. It describes security resource constraints that restrict method invocations in the component. If the resource constraints are satisfied before method transition, the method invocation will be secure. Otherwise, the invocation will be insecure, and potential vulnerabilities will exist in the component.
Through analyzing the normal requirement specification on component security described by XML, the constraint information of interface parameters and environment can be obtained, which is conducive to fault injection testing.
3.3 Detection algorithm of component vulnerability
The FCVD can inject the corresponding faults into the tested component by using the fault injection model when the tested component is driven, and then further trigger component security vulnerability. At the same time, dynamic monitoring mechanism can record the invocation sequence of component method and the running state of component. After a testing task is finished, the framework can analyze the testing report for component vulnerability by using the detection algorithm of component vulnerability. The detection algorithm of component vulnerability is shown in Algorithm 1.
Algorithm 1 Component vulnerability detection (CVD) on component.
Stipulation: R expresses a record in dynamic monitoring log.
Input: Dynamic monitoring log (Initial value is empty); and security requirement specification.
Output: Testing report.
01 {
02 Injecting interface faults and environment faults according to security requirement specification.
03 Driving the tested component.
04 Monitoring invocation sequence of method and exception information, writing to security log.
05 Scanning records in security log one by one.
06 While(Security log has unprocessed R) do
07 {
08 If (Having the exception information in R) then
09 Existing explicit security exception, writing to testing report.
10 If (Method invocation sequence in R violates the security restraints) then
11 Existing implicit security exception, writing to testing report.
12 Marking R as processed record.
13 If (Testing report is not empty) then
14 Outputting the testing report.
15 Go to next unprocessed R in security log.
16 } // end 07
17 } // end 01
In order to theoretically analyze algorithm complexity, we assume that the security log file records one tested component, and each tested component averagely has n methods in a certain period of time. Each component method is compared with relevant component method once. The algorithm complexity is O(n2).
4 Experimental analysis
In order to test the effectiveness and performance of the proposed strategy, we implemented the strategy in component security testing system (CSTS) platform, which was discussed in other paper, and then carried out lots of experiments. The experiment was performed in C# language based on Windows XP operating system and Visual Studio NET 2005 development environment on the PC (personal computer) with memory of 1 GB and CPU of 1.8 GHz. The general testing steps are as follows: creating test project, selecting tested component, analyzing interface information, generating test-file, editing and compiling test fixtures, running test driver, monitoring running process, injecting faults, starting fault injection testing, ending testing and saving project.
We collected 38 commercial components, in which security vulnerabilities existed for experimental analysis in the common vulnerabilities and exposures (CVE) and other security Web sites. At first, we purposely injected some faults into the tested component, and then recorded the security log and exceptional information by dynamic monitoring mechanism. We can select a fault type from the six types of faults for injection testing, and can also select multiple combinations of fault types. Generally speaking, the combination of fault types can trigger component exceptions more easily than a single fault. The experimental results are shown in Table 2. The faults combination of three factors triggers the largest component exception with the detecting rate of 97.37%. The exceptional number of the components triggered by the faults combination of two factors is smaller than that that triggered by the faults combination of three factors, and the detecting rate of the former is 73.70%. The component exceptions triggered by the fault injection of single factor are the least with a detecting rate of 39.47%. From further analysis, we can see that the faults of interface parameters account for 71.6% (see Fig.4) in all the faults that trigger the component exceptions. Most component exceptions are due to buffer overflow that is triggered by long input parameters of method interface. Memory fault is the second important factor to trigger component exceptions, which accounts for 15.2%. Fig.4 shows the proportion of six fault types that trigger component exceptions. In addition, the components of explicit exception triggered account for 83.5%, while the components of implicit exception (violating the security restraints) triggered only account for 16.5%. These components with exceptions triggered by injected faults, have potential security vulnerabilities. The experimental results show that the approach in this work has good feasibility and vulnerability detecting rate.
Table 2 Testing result of K factors fault injection
Fig.4 Detecting rate of six fault types
5 Conclusions and future work
(1) In the component-based software development and maintenance activities, it is a necessary step to test the reliability and security of the third-party component. The testing activity is also easily neglected. Research on an effective testing strategy of component security is an urgent task in the component-based software engineering.
(2) A testing strategy of component security is proposed based on fault injection model (FCVD), and then the fault injecting model, testing framework, and requirement specification of component security is defined and discussed. A detecting algorithm named CVD is also presented for detecting component vulnerabilities.
(3) The testing strategy FCVD mainly has the following advantages: powerful function in fault injection (the faults are injected into the tested component from static level to dynamic level to trigger component exceptions); comprehensive requirement specification of component security (it not only records the component interface methods and properties, but also describes the security constraints of interface methods); nice operability; and effectively detecting COM component security vulnerabilities.
(4) The shortcoming of the strategy is that the security analysis mechanism cannot define the evaluation criterion of security level in details. There will be much important work in future research. Firstly, the approach in which test cases are automatically produced by fault injection model should be proposed. Secondly, some evaluation mechanisms after testing should be worked out to evaluate the security level in details. Finally, a vulnerability database of component should be established based on fault injection model.
References
[1] MAO Cheng-ying, LU Yan-sheng. Research progress in testing techniques of component-based software [J]. Journal of Computer Research and Development, 2006, 43(8): 1375-1382. (in Chinese)
[2] MCGRAW G. Software security [J]. IEEE Security and Privacy, 2004, 2(2): 80-83.
[3] MCGRAW G, ALLEN B. Software security testing [J]. IEEE Security and Privacy, 2004, 2(5): 81-85.
[4] JU A, WANG A. Security testing in software engineering courses [C]// Proeedings of the 34th ASEE/IEEE Frontiers in Education Conference. Los Alamitos, CA: IEEE, 2004: 13-18.
[5] HAN J, ZHENG Y. Security characterisation and integrity assurance for component-based software [C]// Proceedings of 2000 International Conference on Softwave Methods and Tools (SMT 2000). Los Alanmitos, CA: IEEE CS, 2000: 61-66.
[6] GUO F, YU Y, CHIUEH T. Automated and safe vulnerability assessment [C]// Proceedings of Annual Computer Security Applications Conference (ACSAC). Minato-ku, Tokyo: IEEE, 2005: 10-17.
[7] NISSANKE N. Component security-issues and an approach [C]// Proceedings of the 29th Annual International Computer Software and Applications Conference (COMPSAC). Minato-ku, Tokyo: IEEE, 2005: 152-155.
[8] BRYANT E, VINOD G, SANJIT A, SOMESH J, THOMAS W. Automatic discovery of api-level exploits [C]// Proceedings of International Conference of Software Engineer (ICSE). Washington, D C: ACM, 2005: 312-321.
[9] ZHONG Q, EDWARDS N. Security control for COTS components [J]. IEEE Computer, 1998, 31(6): 67-73.
[10] BERTOLINO A, POLINI A. A framework for component deployment testing [C]// Proceedings of the 25th International Conference on Software Engineering (ICSE). Washington, D C: IEEE Computer Society, 2003: 221-231.
[11] HADDOX M J, KAPFHAMMER M G, MICHAEL C C. An approach for understanding and testing third party software components [C]// Proceedings of Annual Reliability and Maintainability Symposium. Los Alamitos, CA: IEEE, 2002: 293-299.
[12] CHEN Jin-fu, LU Yan-sheng, XIE Xiao-dong, ZHANG Wei. Testing approach of component security based on dynamic monitoring [C]// Proceedings of the 2nd International Multi-Symposiums on Computer and Computational Sciences IMSCCS 2007. Los Alamitos, CA: IEEE Computer Society, 2007: 381-386.
[13] LU Yan-sheng, CHEN Jin-fu, XIE Xiao-dong. Testing model of component security based on dynamic monitoring [C]// Proceedings of China National Computer Conference. Beijing: Tsinghua University Press, 2007: 85-92. (in Chinese)
[14] THOMPSON H, WHITTAKER J, MOTTAY F. Software security vulnerability testing in hostile environments [C]// Proceedings of the 2002 ACM Symposium on Applied Computing. Washington, DC: ACM, 2002: 260-264.
[15] DU W X, MATHUR P A. Testing for software vulnerability using environment perturbation [J]. Quality and Reliability Engineering International, 2002, 18(3): 261-272.
[16] HSUEH M, TSAI TK, LYER K R. Fault injection techniques and tools [J]. IEEE Computer, 1997, 30(4): 75-82.
[17] VOAS J. Fault injection for the masses [J]. IEEE Computer, 1997, 30(12): 129-130.
[18] VOAS J, MCGRAW G. Software fault injection: Inoculating programs against errors [M]. New York: John Wiley and Sons, 1997.
[19] LOOKER N, MUNRO M, XU J. A comparison of network level fault injection with code insertion [C]// Proceedings of the 29th IEEE International Computer Software and Applications Conference. Los Alamitos, CA: IEEE, 2005: 479-484.
[20] WHITTAKER A J. Software’s invisible users [J]. IEEE Software, 2001, 18(3): 84-88.
[21] CHEN Ji-feng, ZHU Li, SHEN Jun-yi, WHAN Zhi-hai. Scheme on automated test data generation and its evaluation [J]. Journal of Central South University of Technology, 2006, 13(1): 87-92.
[22] LI Jun-yi, GONG Hong-fang, HU Ji-ping, ZOU Bei-ji, SUN Jia-guang. Class hierarchical test case generation algorithm based on expanded EMDPN model [J]. Journal of Central South University of Technology, 2006, 13(6): 717-721.
[23] JABEEN F, JAFFAR-UR-REHMAN M. A framework for object oriented component testing [C]// Proceedings of the 2005 International Conference on Emerging Technologies. Minato-ku, Tokyo: IEEE, 2005: 451-460.
Foundation item: Project(513150601) supported by the National Pre-Research Project Foundation of China
Received date: 2008-08-27; Accepted date: 2008-10-11
Corresponding author: CHEN Jin-fu, PhD; Tel: +86-27-87556601; E-mail: cjfnet@126.com
(Edited by CHEN Wei-ping)
