模糊测试工具（1）——Defensics

defensics是一个模糊测试工具，百度百科对模糊测试（Fuzzing）的定义是，是一种通过向目标系统提供非预期的输入并监视异常结果来发现软件漏洞的方法。它是通过异常输入的方式触发原来未知的漏洞。
模糊测试的测试用例生成方式有两种：基于生成和基于变异的。基于变异的模糊测试，使用实际的输入，通过随机修改样本或基于样本结构的方式生成测试用例。而基于生成的模糊测试中，需要对被测协议或文件格式有较好的理解，对被测协议或文件格式建立起模型，然后模糊测试工具根据模型生成测试用例，对协议功能有一个完美的覆盖。
defensics公司的一个文档中曾经在测试用例，执行时间和发现漏洞等方面对基于生成和基于变异的模糊测试进行了对比，基于生成的测试执行时间要短很多，测试用例数量要少很多，但是发现的漏洞数却要多很多。
公司利用defencisc主要是做基于TCP/IP协议模糊测试。现阶段主要是利用Condenomicon提供的测试suite来进行的，还未针对公司内部的协议做模糊测试。
defensics是由Codenomicon 科诺斯公司提供的，它也提供了常见协议的test suite，可以到公司网站下载最新的test suite，目前用得比较多的是ARP Server/Client Test Suite，IPv4 test suite，TCP for IPv4 Server/Client Test Suite,ICMPV4 Test Suite等test suite。最多的一个test suite可能有上百万个测试用例，最少的也得上万个。
defensics主要是针对协议，输入一些变异的，异常的，不合法的协议数据，从而检验系统对这些异常数据的处理能力。比如将TCP协议的端口字段增加一个字节，TCP协议的窗口长度为0等，每个test suite是针对一个协议的各个方面的异常数据的总和。
如TCP for TPV4 server test suite的组织结构为：
   TCP for TPV4 server test suite：
       active-close   --named group
       wait-close
       SYNACK-Reset
       Established-Reset
       Sockstress
       known-Attacks
       
已经出现过的漏洞的地方往往还存在脆弱点。
如果不确定当前被测试系统是否能支持test suite或者group，可以自动扫描是否支持。
如果不用官网提供的test suite，自己很难写出比较全面的系统的模糊测试用例。
Defensics也是一个较好的自动化测试工具，所有的测试用例从运行到结果的显示到report都是自动化的，一个suite的测试用例多达百万个，如果不能全自动的运行，也是一件特别伤脑筋的事情。
Defensics可以记录不用level的log。如valid case and failed case，valid case and anomally messages，debug trace（full logging），no trace等，一般选择valid case and failed case，如果全部log都保存下来也需要不少空间啊，关键是分析也费劲。
测试用例运行完成之后，在result里面可以看到，如果测试用例全部通过，则文件夹标记为绿色，如果有测试用例没有通过，这文件夹标记为红色。
 
result文件夹里主要有三个文件，main.log, notes.xml和statistics.csv。main.log记录的是被测系统和测试系统之间的数据包，点开可以看到每个数据包的具体数据。statistics.csv里面会显示pass和fail的测试用例，点开fail的测试用例，也可以看到数据包并了解数据包是在哪里发生了变异。
 
以下是defensics 11 工具官方给出的main log主要包括哪些内容
 Defensics 11 Main log
 
Main - Result view - Main log
Main log
Main log contains detailed test run results, including details of sent and received messages. Main log file is opened to main log viewer that can show details of the file in the viewer component on the right.
Please note that the main log may be a large file in long test runs, depending on the logging settings.
A main log can have the following log entries:
Test case info
The beginning and the end of each test case is logged. Click the test case index line to go to test case documentation. Each test case is given a verdict, usually with some remarks explaining the cause of the verdict.
Messages
Green message line indicates an outgoing message and blue line an incoming message. Message contents may be available by clicking the message line. Use the  'Save to file...'  link to save the message content. For test cases with long overflow anomalies, the content is abbreviated for performance reasons.
Other entries
Test suite specific log entries may be present.
 
note.xml是一个xml格式的report，里面主要是记录了测试系统以及测试suite等信息。对分析具体的漏洞我觉得并没有多大用处。
 
 Defensics 11 Notes file
 
Main - Result view - Notes file
Notes file
Notes can be used to store user-supplied information about a test run. Carefully filled notes may improve test run identification and search remarkably. Notes are stored as an XML file notes.xml. Contrast to other files, which are generated by the test driver during the test run, notes file can be edited by user during and also after the test execution.
Information in notes file is organized into key-value pairs. There are some built-in keys, which are used in report generation. However, user can provide any new keys for own use. The built-in keys are described here.
 
Special directory for results
A directory where to test runs result files are placed. Directory structure of result directories with a defined testplan.directory is: result directory/<testplan directory>/suitename/timestamp/<result files>. Key used in xml file: Testplan.Directory.
 
System under test
Name of the System Under Test (SUT). Added to report document. Key used in xml file: SUT.Name
 
Version of the system under test
Version of the System Under Test (SUT). Added to report document. Key used in xml file: SUT.Version
 
Tester name
Name of the tester. Key used in xml file: Tester.Name
 
Tester contact information
Contact information of the tester, such as e-mail address. Key used in xml file: Tester.Contact
 
Name of the test run
Name of the test run is shown in result browser. This will replace the default test run name consisting of date and time. Key used in xml file: Testrun.Name
 
Report name
Name of an generated report document. Key used in xml file: Report.Name
 
Report information
Longer description added to the Executive summary page of a report document. Key used in xml file: Report.Info
 
Used sequence files
Sequence used in the test run. Visible in result browser. Key used in xml file: Testrun.Used.Sequence
 
Continuation of defined testrun
Denotes this test run is resumed from a previous test run defined by the value. Key used in xml file: Testrun.Continuation-of
 
Continues in defined test run
This test continues in test run defined by the value. Key used in xml file: Testrun.Continued-in
 
Auxiliary test run in
Test run has an auxiliary test run in. Key used in xml file: Testrun.Auxiliary-in
 
Type of the test run
Type of test is marked here, possible values are   normal, auxiliary and valid  . Key used in xml file: Testrun.Type
 
Test case selection mode
Test case selection mode is marked here. Possible values are   all, random, %value, first and last  . Key used in xml file: Testrun.Selection.Mode
Notes file is lazily created by GUI when required. The file does not exit, if there has been no need for it!
Templates and Editing
You can edit notes using result browser by clicking the file notes.xml. You can edit values for built-in keys. You can add new keys as well. Changes to the notes are automatically saved.
You can import notes from templates as well. There is a list of saved templates on top of the view. Mouse over a template will show the saved template content. Select a template and click   "Import from template"  . It will overwrite all the editable values. New templates can be defined in   'All' -> 'Notes'   tab of the GUI.
Notes are saved as xml and are editable in external editors too. See more details about the notes xml format
 
还有一个文件是statistics.csv，这个文件就是以表格的形式记录了测试用例通过与否等信息。官方文档有点长我就不贴了。
 
运行完成之后，还可以利用工具生成失败测试用例的一个summary的report。以下是一个示例。
 
 Test run summary   
 
20140901-0317-03 : TCP for IPv4 Server Test Suite
 
  Overall verdict   
  Overall verdict      FAIL
[   Test case count   ]   1
  Failures    
Verdicts from valid case or external instrumentation
  Analysis tools   
 
  System under test   
  Name   
 
  Version   
 
  Instrumentation methods   
  Valid case instrumentation       ENABLED   
  External instrumentation           DISABLED   
  SNMP instrumentation              DISABLED   
  Instrumentation fail limit           1
  Instrumentation frequency        1
 
  Verdict from valid case instrumentation / connection instrumentation   
  Overall verdict           fail
  Test cases in total    1
  Failed                          1
Passed                        0
  Test execution time   
  Test run started        20140901 03:17:03
  Test run ended         20140901 03:17:13
  Running time           00:00:10
  Average cases per second   
 
  Test setup   
  Name of the tester   
 
  Contact information   
 
  Operating system    Linux i386 3.7-trunk-686-pae
  Java                          1.7.0_25 23.25-b01 mixed mode
   Test suite   
  Name                      TCP for IPv4 Server Test Suite
  Version                  4.2.1
  License                 Licensed to***
  Suite hash            ***
  Options   
  Sequence                                 TCP with HTTP GET payload (in file user/http.seq)
  Test case selection mode      all
  Test run type                             normal
  Options in detail    
...
  Test run analysis   
  Click the links below to perform some advanced analysis of the selected test runs:   
  Denial of Service Analysis    
  Analyze the Denial Of Service (DOS) situations during the failed test cases. The analysis provides an estimate of the vulnerability of SUT for DOS attacks.   
  Response Analysis    
  Find our all different Status responses from the SUT and list them with representative test cases. The analysis provides an overview for SUT behavior and error modes.   
  Slow Test Case Analysis    
  Find the non-failed test cases which running times compared to amount of sent traffic are the longest.   
 
点开  Denial of Service Analysis 链接之后，里面有更详细的在这个测试用例时是否遭受了DoS攻击的信息。
Response Analysis   列出了被测系统SUT在失败的测试用例执行时返回的不同状态。
 
看似测试已经完成，报告也有了，其实工作也许做了1/3不到。
 
1.对所有失败的测试用例得重新执行一遍。
 
2.检查每个测试用例失败时，被测系统的状态并检查这种状态对于系统来说是否是可以接受的以及这种状态带来的潜在的风险是什么。
 
3.会导致DoS的测试用例，可以再利用DoS工具再测试一下。
 
对所有失败的测试用例进行分析统计，就可以了解协议在哪些方面的畸形数据处理方面容易出错从而造成严重的安全问题。最后提交测试报告，提交bug，跟踪bug状态，开发修复之后还得验证bug等等。