This file is a compilation of the few various KeyLabs text files found in the tarball. These are made as one text file for quick and easy online viewing. The text files which come with the testing programs themselves are in separate folders in the tarball. The various text files are separated with lines like: >>> BEGIN SERVER/WORKSTATION WORKSHEET <<< >>> END SERVER/WORKSTATION WORKSHEET<<< >>> BEGIN TTCP TEST TOOL TEXT FILE <<< >>> END TTCP TEST TOOL TEXT FILE <<< The text files included in this file are: ser_wks_cert.txt bonnie_test.txt curl_test.txt miniterm_test.txt smp_test.txt ttcp_test.txt If you have questions you can direct them to linux@keylabs.com. >>> BEGIN SERVER/WORKSTATION WORKSHEET <<< Server/Workstation WorkSheet KeyLabs Server/Workstation Certification Test, Rev. 1.5 - JCW Tester Name: ________________________________________ Date: __________________ Section 1: Product Information Linux Distribution(s) tested: ________________________________________________ ______________________________________________________________________________ Type of Product Test: Server Workstation Manufacturer (Company): ______________________________________________________ Product Name and Model Number: _______________________________________________ BIOS Revision Information: ___________________________________________________ MainBoard and Chipset ________________________________________________________ Processor (List the Number/Type of Processor(s)): ____________________________ Memory Size (RAM) ____________________________________________________________ Host Bus Adapter (if applicable) _____________________________________________ Motherboard Features and Capabilities ________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ Describe the Display Adapter (Model, RAM): ___________________________________ Number and Type of Serial Ports (usually 16550a): ____________________________ Number and Type of Parallel Ports (usually bi-directional): __________________ Number and Type of USB Ports (USB 1.0 or 2.0):________________________________ Number of I/O Ports / Buses: ISA PCI AGP Other: ______________________________ Network Interface Card: ______________________________________________________ CDROM: IDE SCSI Make/Model/Other: ____________________________________________ Sound Chipset / Soundcard: ___________________________________________________ Floppy Drives: Standard 3.5" Make/Model: _____________________________________ Hard Drive(s): IDE SCSI Make/Model:________________________________________ Keyboard Make/Model and Interface: ___________________________________________ Pointing Device (Mouse) Make/Model and Interface: ____________________________ ______________________________________________________________________________ Miscellaneous Hardware _______________________________________________________ ______________________________________________________________________________ Section 2: Product Setup and Testing Note any hardware that the distribution failed to detect: ____________________ ______________________________________________________________________________ ______________________________________________________________________________ Note additional items (modules, drivers, etc) required for support of hardware on distribution(s): __________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ How did you set up devices (w/modules, etc) that weren't detected? ___________ ______________________________________________________________________________ ______________________________________________________________________________ ______________________________________________________________________________ Did any devices require experimental drivers (describe)? _____________________ ______________________________________________________________________________ ______________________________________________________________________________ List special kernel options that should be set: ______________________________ ______________________________________________________________________________ ______________________________________________________________________________ List kernel SCSI support or other device support options that need changed: ______________________________________________________________________________ ______________________________________________________________________________ 1. Perform a CDROM installation: PASS FAIL 2. Network Install (FTP/NFS): PASS FAIL 3. Device AutoProbing: PASS FAIL 4. Kernel Compatibility: PASS FAIL 5. LILO Compatibility: PASS FAIL 6. Heavy Disk I/O Testing: (server only) PASS FAIL 7. Network Testing Ttcp Test: PASS FAIL Curl Test: (server only) PASS FAIL 8. SMP testing: PASS FAIL 9. Display Adapter Console Compatibility: PASS FAIL X Server Compatibility: PASS FAIL 10. Sound Adapter (Workstation only) Basic Sound Compatibility: PASS FAIL Advanced Sound Availability: PASS FAIL Advanced Sound Compatibility: PASS FAIL 11. Serial Ports Kernel Detection: PASS FAIL SetSerial Compatibility: PASS FAIL Throughput Testing: PASS FAIL 12. Parallel Ports Kernel Detection: PASS FAIL Compatibility Testing: PASS FAIL 13. Pointing Device Kernel Detection: PASS FAIL Compatibility Testing: PASS FAIL 14. Keyboard Compatibility Testing: PASS FAIL Section 3: Test Procedures CDROM Install Test Procedure: Tester installs the distribution onto the system's hard disk using the install procedure provided by the distribution. Tester then shutdowns the system down completely (no power to CPU) and restarts the system. Determination: If the install procedure completes and the new system can be booted up from the target hard drive and then operated without further modifications, the test is a pass. Network Install Test Procedure: Tester installs the distribution onto the test system's hard disk via remote connection, whether NFS or FTP. Determination: If the install procedure completes and the new system can be booted up from the target hard drive and operated without further modifications, the test is a pass. Device AutoProbing Check Procedure: Tester reviews dmesg to verify the distribution correctly detected the following items if they are included in the test system: videocard, sound card, NIC card, mouse, parallel port, serial ports, processors, RAM, all connected disks (fixed and removable, including floppy) AND correctly identified the partitions on the drives. Determination: If all connected hardware is properly detected and identified, the test is a pass. Failure to detect and identify any of the items listed in the Procedures area listed above is a failure. Kernel Compatibility Check Procedure: Tester verifies the features of the system being tested can be used with the kernel provided with the distribution. Determination: If a kernel modification and re-compile is necessary, the tester should note this fact for potential users, but simply needing to modify and re-compile is not a failure. Failure is based on inability of the system to work at all with the supplied kernel. Grub / Lilo Compatibility Check Procedure: Tester verifies the system can be booted using Lilo or Grub after the install process is complete. Determination: If the system as installed cannot be booted directly to the hard disk without using either a book diskette or the CDROM, the test is a fail. If the system can be booted directly to the hard disk, the test is a pass. If the tester must first boot using a boot disk or CDROM, make changes to lilo.conf and re-run lilo to get the system to boot, this is not a failure but must be noted for inclusion in the final report. Heavy I/O (bus) Test (Server Only) Procedure: Tester follows the procedures for the bonnie load test as included in the README inside the bonnie tarball. Determination: If the system continues performing the operations for the specified amount of time (three hours minimum) the test is a pass. If the system locks, prematurely terminates or otherwise stops the test, the test is a fail. Network Testing Ttcp Test Procedure: Tester configures, installs and runs the ttcp-based test as specified in the ttcp_test.txt file. Determination: The connection must be made and transfer rates must be within ten percent (10%) of the manufacturer's (manufacturer of the NIC) maximum transfer rate for the test to be a pass; otherwise, fail. Curl Test (server only) Procedure: Tester configures, installs and runs the Curl-based network test as specified in the curl_test.txt file. Determination: If the server continues performing the operations for the specified amount of time (eight hour minimum) then the test is a pass. If the system locks, prematurely terminates or otherwise stops working, the test is a fail. SMP Test (Multi-processor machines only) Procedure: Tester configures and installs the test tools according to the smp_test.txt file found in the key_serwks.tar.gz tarball. Determination: If the server shows proper operation as specified in the smp_test.txt file the test is a pass. Otherwise, if the processors don't work properly then the test is a fail. Display Adapter Compatibility Test Console Compatibility Test Procedure: Tester operates the system using the console to verify the video display adapter will setup and work correctly. Tester must verify proper keyboard functionality (use an editor such as Emacs) and proper mouse functionality (have gpm installed and running.) Determination: All fonts must show correctly; keys must funcitons properly inside an editor; control keys must function properly. For gpm tests, all three buttons must work correctly when using the point/cut/paste functions. X Server Compatibility Test Procedure: Tester uses the distribution's preferred method of setting up an X server. Tester starts the X server and verifies the mouse, keyboard and video display work properly. If there is any doubt about the functionality of any one of these, the tester is to run the X-related tests found in the KeyLabs Video Card Certification Test. Tester should make comprehensive notes if there are any problems and what solutions were found/devised. Determination: All features of control keys, keyboard and mouse must work properly. Screen resolutions must be extant and accessable. X server must start and be stopable without any adverse effects to the rest of the system (specifically, lockups.) If all control feature of control keys, keyboard and mouse for correctly; if screen resolutions are accessable and the X server can be started and stopped without locking the system the test is a pass; otherwise, fail. Sound Adapter Compatibility Test Basic Sound Compatibility Procedure: Tester verifies the most basic sound abilities of the system work on the base Linux install by having the pc speaker make a beep -- any beep will do provided the command to beep comes from the Linux OS or a program running in the Linux OS environment. The tester can use the console_beep program (found in the testing tools area on the KeyLabs web site) if needed. Determination: If the pc speaker makes a beep, the test is a pass; otherwise, fail. If the system being tested is promoted as having sound capabilities beyond a simple pc speaker, the tester should advance to the next step in the Sound Adapter Compatibility Test (Advanced Sound Availability.) Advanced Sound Availability Procedure: Tester checks with both the system's manufacturer and the web to determine if the system's sound system can be used with Linux and has available drivers. Determination: If the manufacturer reports there are no sound drivers for the system and if the Linux sound-related websites reveal no drivers available for the system, then the tester answers "NO" to this question and does not need to perform the Advanced Sound Compatibility Test. Advanced Sound Compatibility Procedure: Tester uses the distribution's preferred method of setting up the sound drivers for the system's sound system. Tester perfoms sound checks by ensuring the following types of sounds can be played and heard. Sound quality must be consistent with expectations for the type of sound being used (CD-ROM music disk sound must be comparable with sound played on any other CD playing system.) Tested sound sources must include: a CD-ROM music disk (only if the system has a CD/DVD disk player); an mp3 sound file; a .wav sound file; a midi sound file; Determination: If the system can play all the above sound file types and the sound is comparable to sound from similar playing device, the test is a pass; otherwise, fail. Serial Ports Tests Procedure: Tester first verifies the serial ports are detected during bootup by checking dmesg. Secondly, tester verifies serial ports can work with setserial (use setserial to set the baud rate). Thirdly, tester tests the serial ports by connecting a null modem, compiles the miniterm program and passes data through the serial ports. Determination: dmesg: serial ports must be properly detected during bootup. Setserial: serial ports must be configurable via setserial. null modem: serial ports must be able to pass data through the modem. Parallel Port Tests Procedure: Tester first verifies the parallel port is detected during the boot process. Tester then connects and configures a printer to the printer port and prints a file. Determination: Tester must be able to print a file from the test unit on the connected printer. A simple dump of an ASCII file to the printer is adequate to test the parallel port. Pointing Device Tests Procedure: Tester first verifies the mouse/pointing device is detected during the boot process (check dmesg.) Tester then uses gpm (server) or X to check the basic pointing and clicking functions required for proper operation of the system. Determination: If the pointing device is properly detected and functions correctly as concerns pointing, clicking, dragging, cut and paste functions with either gpm or both gpm and X, then the test is a pass. Failure of any one of these items is a fail for the entire Pointing Device test. Keyboard Device Tests Procedure: Tester loads a text editor or word processor and verifies the operation of the keyboard. Tester must check all features of the keyboard including control keys, shift keys, etc. Keyboard should be checked in both the X environment and the console environment. Determination: If the keyboard functions properly, the test is a pass. Otherwise, the test is a fail. >>> END SERVER/WORKSTATION WORKSHEET <<< >>> BEGIN BONNIE TEST <<< bonnie_test.txt for KeyLabs Test Packages Bonnie is a file system benchmarking utility. It is used for identifying and measuring file system bottlenecks. It is also useful for stress testing the file system since it can put a heavy load on a file system. Using bonnie ------------ Steps: 1. Put bonnie in your path (/usr/local/bin/ is good) 2. Set the BONNIESIZE variable (see below) 3. Execute the run-bonnie script Steps Explained --------------- To use bonnie you'll first want to make sure bonnie is somewhere in the path so the system can find and run it. One of the common bin directories is good (/usr/local/bin/) or your personal bin directory (~/bin/) will work too. The run-bonnie script, obviously, requires the bonnie disk test to be installed and in the path. The run-bonnie script is a simple script to file used to run the "bonnie" disk test over and over. You need to set up the BONNIESIZE environment variable before running this script. Multiply the ammount of system memory by 2.5 and execute tne following, wnere n equals your desired test size: export BONNIESIZE=n You can check the BONNESIZE variable with this command: echo $BONNIESIZE If you have problems getting bonnie or run-bonnie to run, check permissions on the two files: Both must be executable. Bonnie will return some information about each test. See the man page for specifics about the output contents and format. >>> END BONNIE TEST TEXT <<< >>> BEGIN CURL TEST TEXT <<< curl_test.txt for KeyLabs Test Packages Curl is used to load test the ftp and Apache servers as well as the network connections of a system. Using the Curl Test ------------------- Steps: 1. Put curl in the path (/usr/local/bin/ for example) 2. copy testfiles into the proper ftp and httpd directories (see below) 3. run the testfiles script once in each directory. 4. Check that all the ftp* and www* files are in the same directory as the startload and stopload script files. Make sure all have executable permissions set. 5. Set the TESTMACHINE environmental variable to the IP address of the machine being tested. 6. Run the startload script to start the loop tests, run the stopload program to stop the loop tests. Steps Explained --------------- Put the curl program somewhere in the path: /usr/bin/ or /usr/local/bin for example. Next you'll need to make the test files used by the test. These are the files which actually get transferred. The file named testfiles is a simple bash script that uses /dev/zero to output the various test files that need to be on the test system. Copy the file testfiles into the pub directory for your ftp installation (often /home/ftp/pub) and also copy it into the html directory of your httpd directory (often /home/httpd/html). Make sure the file named testfiles is executable the run it once in each of the directories and the test files will be generated. The load testing scripts were developed using curl 5.5.1, but newer versions should work as well. These are the script files with the names ftp* and www*. They need to be in the directory from which you will run the startload and stopload script files. These files will also have to have executable permissions set. Set the TESTMACHINE environmental variable to the IP address of the machine being tested. For example: export TESTMACHINE=192.168.0.14 The startload script file will run all 22 load test loops while the stopload script file will stop all load test loops. Be sure to check the stopload program for executable permissions before you run startload. >>> END CURL TEST TEXT <<< >>> BEGIN MINITERM TEST TEXT <<< Testing Notes: Testing the serial ports 1. About this document This document describes the testing of serial ports under Linux. This test is not by any means exhaustive -- it is used to simply verify the serial ports work on a given machine. To do this test, you'll need to have either has two computers with working serial ports or a single computer with two serial ports. You will also need one null-modem serial cable to go between the two serial ports to be tested. 2. Making the miniterm_ttys* programs from scratch This KeyLabs package comes with both the source code and compiled executables for the miniterm_ttys0 and miniterm_ttys1 programs. If you don't have a copy of miniterm.c and you want the whole coconut, you can get it at: http://metalab.unc.edu/pub/Linux/docs/LDP/programmers-guide/lpg-0.4.examples.tar.gz. The miniterm.c program is one of the examples. Download and save the tarball above to it's own directory and uncompress. tar -xvzf lpg-0.4.examples.tar.gz This will create an examples directory where you'll find a file called miniterm.c. This is the only file you will need if you want to do these tests. This editing example assumes you have two standard serial ports in your machine and they are setup for ttyS0 ttyS1 (com1 & com2 in DOS lingo.) Using your favorite text editor open miniterm.c. Modify the modem definition to ttyS0: #include #include #include #include #include #define BAUDRATE B38400 #define MODEMDEVICE "/dev/modem" <=====Change this to "/dev/ttyS0" Save the file without closing your text editor. Compile the saved miniterm.c code. gcc -o miniterm_ttys0 miniterm.c. This will produce an executable file called miniterm_ttys0. Now repeat the process for ttyS1. Again, change the definition, save and compile with this command string: gcc -o miniterm_ttys1 miniterm.c This will produce an executable file called miniterm_ttys1. Depending on how the testing system is setup during install, you may need to change the permissions on the serial port devices to allow data traffic. chmod a+rw /dev/ttyS0 chmod a+rw /dev/ttyS1 3. Running the test Steps: 1. If you don't have the miniterm_ttys* programs, make them as described above. 2. Connect the serial ports to be tested according to how you're going to test. (See Same Machine or Different Machine tests below.) 3. Run the miniterm_ttys* programs as described below. There are two ways you can run this test; either on the same machine with a null modem connecting the two serial ports or across two different machines with a null modem connecting the serial ports to be tested. KeyLabs testers usually run this test across the serial ports of two separate machines. Same machine test Connect a null modem cable to the two serial ports. Next, either open two xterms on your desktop or switch to a different vc. In the terminals, cd into the directory which contains the two programs you just compiled. In one term execute miniterm_ttys0 and in the other execute miniterm_tys1. If the connections are correct you should see anything typed in one term window show in the other term window. Across different machines test Connect a null modem cable to a serial port on each machine. Next, either open an xterm or a vc on each machine. In the terminals, cd into the directory which contains the two programs you just compiled. In one term execute miniterm_ttys0 and in the other execute miniterm_tys1. If the connections are correct you should see anything typed in one term window show in the other term window. You may need to try various combinations of the miniterm programs on each machine until you find which one is connected to ttyS0 and ttys1. Once the connections are correct you should see anything typed in one term window show in the other term window. >>> END MINITERM TEST TEXT <<< >>> BEGIN SMP TEST TEXT <<< smp_test.txt file for KeyLabs Test Packages About this SMP Test ------------------- The purpose of this test is to verify the CPUs in a test system are working. The test itself is a very small program called loop5d, and it will make each CPU in the system work very hard for a short period of time. The test is currently in various pieces but will be collected into one cohesive chuck of code shortly. The loop programs are from Cameron MacKinnon (email address listed in the code below). The website at which this code can be retrieved is: http://www.phy.duke.edu/brahma/benchmarks.smp These loop programs do the actual work. The SMP_test script program is a way to ease getting the loops working and is not necessary to the test, but can be convienient at times. The xosview program provides the tester with a visual guage for observing the CPUs at work. This test was assembled from information provided by the SMP-HOWTO: http://www.irisa.fr/prive/dmentre/smp-faq/ Setting up for testing - loop5d program and SMP_test script ----------------------------------------------------------- [Note: The following describes use of the loop5d.c code but the tester can use any of the loop snippets he chooses.] 1. Compile the loop5d.c program with the following command line: gcc -Wall06 -o loop5d loop5d.c This will give you an executable program named loop5d 2. Edit the SMP_test script for number of CPUs on the test system. For each CPU you need one instance of "time loop5d" For example: Two CPUs in test machine: time loop5d & time loop5d Four CPUs in test machine: time loop5d & time loop5d & time loop5d & time loop5d {Note: Yes, I know this is some serious cheese. I'm working on it when time allows.] 3. Check the permissions on both the loop5d program and the SMP_test script for executability. 4. Copy the loop5d program into the path (/usr/local/bin) Setting up for testing - xosview program. ---------------------------------------- 1. Copy the xosview tarball from the floppy to the harddisk: cp /floppy/xosview*.tgz /usr/local/ 2. Change directory to /usr/local/ and Untar the xosview tarball: cd /usr/local/; tar -xvzf xosview*.tgz 3. Enter the xosview directory: cd xosview*/ 4. Configure the xosview program for make: ./configure 5. Make and install the program: make; make install Running the test ---------------- 1. Start xosview in the background: xosview & 2. Start SMP_test: ./SMP_test Monitoring Results ------------------ Watch the CPU indicators, they'll usually peg out on the far right. For every CPU indicator, their should be a solid green line almost to the very end of the indicator. The test loop only runs for a short period of time (usually less than three minutes for the loop5d program.) Don't be concerned with exit errors from the loops -- we're just making sure the CPUs are working. The test is a pass if all CPU indicators show the CPUs are working and working at approximately the same time. Note that not all the test loops found in the /loop directory will make all the CPUs on the machine work at the same time. See Cameron's website (listed above) for detailed info about the various loop tests. Loop5d Test Code ---------------- The test code for loop5d is included here just in case. Instructions for compiling: gcc -Wall06 -o loop5d loop5d.c --- Begin code for loop5d.c --- /* Code by: with (very) minor mods by James C. Williams, KeyLabs Linux Team. Case 10b: Real World(tm) floating point with optimized compile [Note: To get the optimized compile, use the following: gcc (or egcs) -Wall06 loop5d.c -o loop5d. This will make a loop5d program in your directory. This code is identical to loop5c.c without the -Wall06.] Command line for case 10 above: time loop5d; time loop5d & time loop5d */ #define LINE 8 main(){ double buff[1000000]; unsigned long int i; int j; for (j=0; j<43*LINE; j++) for (i=0; i<1000000; i+=LINE/sizeof(double)) buff[i]=buff[i]*3.14+2.71; } --- End code for loop5d.c >>> END SMP TEST TEXT <<< >>> BEGIN TTCP TEST TEXT <<< ttcp_test.txt file for KeyLabs Test Packages About the test -------------- The ttcp test shoots a file across a network connection while providing some feedback in the form of statistics. This information can be used to compare with performance statistics provided by manufacturers of network components to ensure the numbers are in the same ballpark. The test uses the ttcp program and two script files which operate the ttcp program. The scripts require ttcp to be installed and in the path and assume you are running the same version of ttcp on both ends -- some versions might use different ports. This test must be conducted between two different machines. Steps: 1. Copy the ttcp program into the path. 2. Put the ttcp script files into a directory on each machine. 3. Set the LABMACHINE environmental variable. (see below) 4. Make the testfile on the transmitting machine. 5. Run ttcp-receiver on the receiving machine. 6. Run the ttcp-transmit script on the transmitting machine. Steps Explained --------------- 1. Put a copy of the ttcp program into the path on each machine being used in the test. The ttcp program must be the same version on both machines. 2. Put the script files ttcp-receive and ttcp-transmit into their own directories on each machine. Check them for executability. 3. You must set up the LABMACHINE environment variable. To do so, execute the following command, substituting the address of the system running the reciever for the 10.10.10.1 address: export LABMACHINE=10.10.10.1 4. Create a file "testfile" to be sent during the test. The easiest way to do this is with dd and /dev/zero where "count" is however many megabytes you wish to use. This shouldn't be too large since it's better if it can fit in the cache. dd if=/dev/zero of=testfile bs=1048576 count=10 5. On the machine receiving the file, run the ttcp-receive program. It will wait a bit for a connection. 6. On the machine transmitting the file, run the ttcp-transmit program and the testfile will be sent. Monitoring the Test ------------------- Each time the file is transmitted, the ttcp program will return a set of statistics on each machine.