From 5026c7f91d23405fe3fc3d5d5618f2f1b9398299 Mon Sep 17 00:00:00 2001
From: Tanya Lattner
To run all of the simple tests in LLVM, use the master Makefile in the +
To run all of the simple tests in LLVM using DejaGNU, use the master Makefile in the llvm/test directory:
% gmake -C llvm/test+or
+% gmake check ++
To run only a subdirectory of tests in llvm/test using DejaGNU (ie. Regression/Transforms). Just substitute the path to the subdirectory:
++% gmake -C llvm/test TESTSUITE=Regression/Transforms ++
To run only the code fragment tests (i.e. those that do basic testing of -LLVM), run the tests organized by QMTest:
+To run the simple tests (i.e. those that do basic testing of +LLVM), using QMTest:
% gmake -C llvm/test qmtest@@ -169,7 +186,7 @@ language front end.
Code fragments are not complete programs, and they are never executed to determine correct behavior.
-Thes code fragment tests are located in the llvm/test/Features and +
These code fragment tests are located in the llvm/test/Features and llvm/test/Regression directories.
@@ -260,6 +277,82 @@ test and database classes. + + + + + +The LLVM test suite is partially driven by DejaGNU and partially +driven by GNU Make. Specifically, the Features and Regression tests +are all driven by DejaGNU (and optionally QMTest). The llvm-test +module is currently driven by a set of Makefiles.
+ +The DejaGNU structure is very simple, but does require some +information to be set. This information is gathered via configure and +is written to a file, site.exp in llvm/test. The llvm/test +Makefile does this work for you.
+ +In order for DejaGNU to work, each directory of tests must have a +dg.exp file. This file is a program written in tcl that calls +the llvm-runtests procedure on each test file. The +llvm-runtests procedure is defined in +llvm/test/lib/llvm-dg.exp. Any directory that contains only +directories does not need the dg.exp file.
+ +In order for a test to be run, it must contain information within +the test file on how to run the test. These are called RUN +lines. Run lines are specified in the comments of the test program +using the keyword RUN followed by a colon, and lastly the +commands to execute. These commands will be executed in a bash script, +so any bash syntax is acceptable. You can specify as many RUN lines as +necessary. Each RUN line translates to one line in the resulting bash +script. Below is an example of legal RUN lines in a .ll +file:
++; RUN: llvm-as < %s | llvm-dis > %t1 +; RUN: llvm-dis < %s.bc-13 > %t2 +; RUN: diff %t1 %t2 ++
There are a couple patterns within a RUN line that the +llvm-runtest procedure looks for and replaces with the appropriate +syntax:
+There are also several scripts in the llvm/test/Scripts directory +that you might find useful when writing RUN lines.
+ +Lastly, you can easily mark a test that is expected to fail on a +specific platform by using the XFAIL keyword. Xfail lines are +specified in the comments of the test program using XFAIL, +followed by a colon, and one or more regular expressions (separated by +a comma) that will match against the target triplet for the +machine. You can use * to match all targets. Here is an example of an +XFAIL line:
++; XFAIL: darwin,sun ++
The LLVM test suite is partially driven by QMTest and partially driven by GNU -Make. Specifically, the Features and Regression tests are all driven by QMTest. -The llvm-test module is currently driven by a set of Makefiles.
+The Feature and Regression tests can also be run using QMTest.
The QMTest system needs to have several pieces of information available; these pieces of configuration information are known collectively as the