Add project files.
This commit is contained in:
278
3rd-party/curl-7.52.1/tests/README
vendored
Normal file
278
3rd-party/curl-7.52.1/tests/README
vendored
Normal file
@@ -0,0 +1,278 @@
|
||||
_ _ ____ _
|
||||
___| | | | _ \| |
|
||||
/ __| | | | |_) | |
|
||||
| (__| |_| | _ <| |___
|
||||
\___|\___/|_| \_\_____|
|
||||
|
||||
The curl Test Suite
|
||||
|
||||
1. Running
|
||||
1.1 Requires to run
|
||||
1.2 Port numbers used by test servers
|
||||
1.3 Test servers
|
||||
1.4 Run
|
||||
1.5 Shell startup scripts
|
||||
1.6 Memory test
|
||||
1.7 Debug
|
||||
1.8 Logs
|
||||
1.9 Test input files
|
||||
1.10 Code coverage
|
||||
1.11 Remote testing
|
||||
|
||||
2. Numbering
|
||||
2.1 Test case numbering
|
||||
|
||||
3. Write tests
|
||||
3.1 test data
|
||||
3.2 curl tests
|
||||
3.3 libcurl tests
|
||||
3.4 unit tests
|
||||
|
||||
4. TODO
|
||||
4.1 More protocols
|
||||
4.2 SOCKS auth
|
||||
|
||||
==============================================================================
|
||||
|
||||
1. Running
|
||||
|
||||
1.1 Requires to run
|
||||
|
||||
perl (and a unix-style shell)
|
||||
python (and a unix-style shell)
|
||||
diff (when a test fails, a diff is shown)
|
||||
stunnel (for HTTPS and FTPS tests)
|
||||
OpenSSH or SunSSH (for SCP, SFTP and SOCKS4/5 tests)
|
||||
nghttpx (for HTTP/2 tests)
|
||||
|
||||
1.2 Port numbers used by test servers
|
||||
|
||||
- TCP/8990 for HTTP
|
||||
- TCP/8991 for HTTPS
|
||||
- TCP/8992 for FTP
|
||||
- TCP/8993 for FTPS
|
||||
- TCP/8994 for HTTP IPv6
|
||||
- TCP/8995 for FTP (2)
|
||||
- TCP/8996 for FTP IPv6
|
||||
- UDP/8997 for TFTP
|
||||
- UDP/8998 for TFTP IPv6
|
||||
- TCP/8999 for SCP/SFTP
|
||||
- TCP/9000 for SOCKS
|
||||
- TCP/9001 for POP3
|
||||
- TCP/9002 for IMAP
|
||||
- TCP/9003 for SMTP
|
||||
- TCP/9004 for SMTP IPv6
|
||||
- TCP/9005 for RTSP
|
||||
- TCP/9006 for RTSP IPv6
|
||||
- TCP/9007 for GOPHER
|
||||
- TCP/9008 for GOPHER IPv6
|
||||
- TCP/9008 for HTTPS server with TLS-SRP support
|
||||
|
||||
1.3 Test servers
|
||||
|
||||
The test suite runs simple FTP, POP3, IMAP, SMTP, HTTP and TFTP stand-alone
|
||||
servers on the ports listed above to which it makes requests. For SSL tests,
|
||||
it runs stunnel to handle encryption to the regular servers. For SSH, it
|
||||
runs a standard OpenSSH server. For SOCKS4/5 tests SSH is used to perform
|
||||
the SOCKS functionality and requires a SSH client and server.
|
||||
|
||||
The base port number (8990), which all the individual port numbers are
|
||||
indexed from, can be set explicitly using runtests.pl' -b option to allow
|
||||
running more than one instance of the test suite simultaneously on one
|
||||
machine, or just move the servers in case you have local services on any of
|
||||
those ports.
|
||||
|
||||
The HTTP server supports listening on a Unix domain socket, the default
|
||||
location is 'http.sock'.
|
||||
|
||||
1.4 Run
|
||||
|
||||
'make test'. This builds the test suite support code and invokes the
|
||||
'runtests.pl' perl script to run all the tests. Edit the top variables
|
||||
of that script in case you have some specific needs, or run the script
|
||||
manually (after the support code has been built).
|
||||
|
||||
The script breaks on the first test that doesn't do OK. Use -a to prevent
|
||||
the script from aborting on the first error. Run the script with -v for more
|
||||
verbose output. Use -d to run the test servers with debug output enabled as
|
||||
well. Specifying -k keeps all the log files generated by the test intact.
|
||||
|
||||
Use -s for shorter output, or pass test numbers to run specific tests only
|
||||
(like "./runtests.pl 3 4" to test 3 and 4 only). It also supports test case
|
||||
ranges with 'to', as in "./runtests 3 to 9" which runs the seven tests from
|
||||
3 to 9. Any test numbers starting with ! are disabled, as are any test
|
||||
numbers found in the files data/DISABLED or data/DISABLED.local (one per
|
||||
line). The latter is meant for local temporary disables and will be ignored
|
||||
by git.
|
||||
|
||||
When -s is not present, each successful test will display on one line the
|
||||
test number and description and on the next line a set of flags, the test
|
||||
result, current test sequence, total number of tests to be run and an
|
||||
estimated amount of time to complete the test run. The flags consist of
|
||||
these letters describing what is checked in this test:
|
||||
|
||||
s stdout
|
||||
d data
|
||||
u upload
|
||||
p protocol
|
||||
o output
|
||||
e exit code
|
||||
m memory
|
||||
v valgrind
|
||||
|
||||
1.5 Shell startup scripts
|
||||
|
||||
Tests which use the ssh test server, SCP/SFTP/SOCKS tests, might be badly
|
||||
influenced by the output of system wide or user specific shell startup
|
||||
scripts, .bashrc, .profile, /etc/csh.cshrc, .login, /etc/bashrc, etc. which
|
||||
output text messages or escape sequences on user login. When these shell
|
||||
startup messages or escape sequences are output they might corrupt the
|
||||
expected stream of data which flows to the sftp-server or from the ssh
|
||||
client which can result in bad test behaviour or even prevent the test
|
||||
server from running.
|
||||
|
||||
If the test suite ssh or sftp server fails to start up and logs the message
|
||||
'Received message too long' then you are certainly suffering the unwanted
|
||||
output of a shell startup script. Locate, cleanup or adjust the shell
|
||||
script.
|
||||
|
||||
1.6 Memory test
|
||||
|
||||
The test script will check that all allocated memory is freed properly IF
|
||||
curl has been built with the CURLDEBUG define set. The script will
|
||||
automatically detect if that is the case, and it will use the
|
||||
'memanalyze.pl' script to analyze the memory debugging output.
|
||||
|
||||
Also, if you run tests on a machine where valgrind is found, the script will
|
||||
use valgrind to run the test with (unless you use -n) to further verify
|
||||
correctness.
|
||||
|
||||
runtests.pl's -t option will enable torture testing mode, which runs each
|
||||
test many times and makes each different memory allocation fail on each
|
||||
successive run. This tests the out of memory error handling code to ensure
|
||||
that memory leaks do not occur even in those situations. It can help to
|
||||
compile curl with CPPFLAGS=-DMEMDEBUG_LOG_SYNC when using this option, to
|
||||
ensure that the memory log file is properly written even if curl crashes.
|
||||
|
||||
1.7 Debug
|
||||
|
||||
If a test case fails, you can conveniently get the script to invoke the
|
||||
debugger (gdb) for you with the server running and the exact same command
|
||||
line parameters that failed. Just invoke 'runtests.pl <test number> -g' and
|
||||
then just type 'run' in the debugger to perform the command through the
|
||||
debugger.
|
||||
|
||||
1.8 Logs
|
||||
|
||||
All logs are generated in the log/ subdirectory (it is emptied first in the
|
||||
runtests.pl script). Use runtests.pl -k to force it to keep the temporary
|
||||
files after the test run since successful runs will clean it up otherwise.
|
||||
|
||||
1.9 Test input files
|
||||
|
||||
All test cases are put in the data/ subdirectory. Each test is stored in the
|
||||
file named according to the test number.
|
||||
|
||||
See FILEFORMAT for the description of the test case files.
|
||||
|
||||
1.10 Code coverage
|
||||
|
||||
gcc provides a tool that can determine the code coverage figures for
|
||||
the test suite. To use it, configure curl with
|
||||
CFLAGS='-fprofile-arcs -ftest-coverage -g -O0'. Make sure you run the normal
|
||||
and torture tests to get more full coverage, i.e. do:
|
||||
|
||||
make test
|
||||
make test-torture
|
||||
|
||||
The graphical tool ggcov can be used to browse the source and create
|
||||
coverage reports on *NIX hosts:
|
||||
|
||||
ggcov -r lib src
|
||||
|
||||
The text mode tool gcov may also be used, but it doesn't handle object files
|
||||
in more than one directory very well.
|
||||
|
||||
1.11 Remote testing
|
||||
|
||||
The runtests.pl script provides some hooks to allow curl to be tested on a
|
||||
machine where perl can not be run. The test framework in this case runs on
|
||||
a workstation where perl is available, while curl itself is run on a remote
|
||||
system using ssh or some other remote execution method. See the comments at
|
||||
the beginning of runtests.pl for details.
|
||||
|
||||
2. Numbering
|
||||
|
||||
2.1 Test case numbering
|
||||
|
||||
1 - 99 HTTP
|
||||
100 - 199 FTP
|
||||
200 - 299 FILE
|
||||
300 - 399 HTTPS
|
||||
400 - 499 FTPS
|
||||
500 - 599 libcurl source code tests, not using the curl command tool
|
||||
600 - 699 SCP/SFTP
|
||||
700 - 799 SOCKS4 (even numbers) and SOCK5 (odd numbers)
|
||||
800 - 849 IMAP
|
||||
850 - 899 POP3
|
||||
900 - 999 SMTP
|
||||
1000 - 1299 miscellaneous
|
||||
1300 - 1399 unit tests
|
||||
1400 - 1499 miscellaneous
|
||||
1500 - 1599 libcurl source code tests, not using the curl command tool
|
||||
(same as 5xx)
|
||||
1600 - 1699 unit tests
|
||||
2000 - x multiple sequential protocols per test case
|
||||
|
||||
There's nothing in the system that *requires* us to keep within these number
|
||||
series.
|
||||
|
||||
3. Write tests
|
||||
|
||||
Here's a quick description on writing test cases. We basically have three
|
||||
kinds of tests: the ones that test the curl tool, the ones that build small
|
||||
applications and test libcurl directly and the unit tests that test
|
||||
individual (possibly internal) functions.
|
||||
|
||||
3.1 test data
|
||||
|
||||
Each test has a master file that controls all the test data. What to read,
|
||||
what the protocol exchange should look like, what exit code to expect and
|
||||
what command line arguments to use etc.
|
||||
|
||||
These files are tests/data/test[num] where [num] is described in section 2
|
||||
of this document, and the XML-like file format of them is described in the
|
||||
separate tests/FILEFORMAT document.
|
||||
|
||||
3.2 curl tests
|
||||
|
||||
A test case that runs the curl tool and verifies that it gets the correct
|
||||
data, it sends the correct data, it uses the correct protocol primitives
|
||||
etc.
|
||||
|
||||
3.3 libcurl tests
|
||||
|
||||
The libcurl tests are identical to the curl ones, except that they use a
|
||||
specific and dedicated custom-built program to run instead of "curl". This
|
||||
tool is built from source code placed in tests/libtest and if you want to
|
||||
make a new libcurl test that is where you add your code.
|
||||
|
||||
3.4 unit tests
|
||||
|
||||
Unit tests are tests in the 13xx sequence and they are placed in tests/unit.
|
||||
There's a tests/unit/README describing the specific set of checks and macros
|
||||
that may be used when writing tests that verify behaviors of specific
|
||||
individual functions.
|
||||
|
||||
The unit tests depend on curl being built with debug enabled.
|
||||
|
||||
4. TODO
|
||||
|
||||
4.1 More protocols
|
||||
|
||||
Add tests for TELNET, LDAP, DICT...
|
||||
|
||||
4.2 SOCKS auth
|
||||
|
||||
SOCKS4/5 test deficiencies - no proxy authentication tests as SSH (the
|
||||
test mechanism) doesn't support them
|
Reference in New Issue
Block a user