By setting limit_to_features to a non empty list of features a configuration can
restrict the set of tests to run to only include tests that require a feature in
this list.
rdar://21082253
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@238766 91177308-0d34-0410-b5e6-96231b3b80d8
These changes allow usages where you want to pass an additional
commandline option to all invocations of a specific llvm tool. Example:
> llvm-lit -Dllc=llc -enable-misched -verify-machineinstrs
Differential Revision: http://reviews.llvm.org/D9487
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@236461 91177308-0d34-0410-b5e6-96231b3b80d8
The sanitizer test suite uses this idiom to disable a test. Now that we
actually check if a test ran after invoking it, we see that zero tests
ran, and complain.
Instead, ignore tests starting with DISABLED_ completely. Fixes the
sanitizer test suite failures on Windows.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234247 91177308-0d34-0410-b5e6-96231b3b80d8
The '/' character in the test name of a type-parameterized test is not a
path separator, and should not be '\' on Windows. We were passing a test
name to --gtest_filter which found no tests, so the exit code was zero,
indicating a passed test.
This bug has been here since r84387 in 2009, when Jeff Yasskin added the
original lit support for type-paratermized tests. Somewhere along the
line some of the ValueMapTests started failing, but we can fix those
separately.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@234242 91177308-0d34-0410-b5e6-96231b3b80d8
The internal shell was already threading around a 'cwd' parameter. We
just have to make it mutable so that we can update it as the test script
executes.
If the shell ever grows support for environment variable substitution,
we could also implement support for export.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@231017 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: I think this is probably a bug, but I'm putting this up for review just to be sure. I think that `lit.util.capture` should decode the resulting string in the same way `lit.util.executeCommand` does.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6769
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225681 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The following types can be encoded and decoded by the json library:
`dict`, `list`, `tuple`, `str`, `unicode`, `int`, `long`, `float`, `bool`, `NoneType`.
`JSONMetricValue` can be constructed with any of these types, and used as part of Test.Result.
This patch also adds a toMetricValue function that converts a value into a MetricValue.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits, llvm-commits
Differential Revision: http://reviews.llvm.org/D6576
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224628 91177308-0d34-0410-b5e6-96231b3b80d8
We were already requiring 2.5, which meant that people on old linux distros
had to upgrade anyway.
Requiring python 2.6 will make supporting 3.X easier as we can use the 3.X
exception syntax.
According to the discussion on llvmdev, there is not much value is requiring
just 2.6, we may as well just require 2.7.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224129 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch gives me just enough to leverage the existing functionality in `TestRunner` for use in `libc++` and `libc++abi` .
It does the following:
* Adds the `UNSUPPORTED` tag to `TestRunner.parseIntegratedTestScript`.
* Allows `parseIntegratedTestScript` to return an empty script if a script is not required by the caller.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits, llvm-commits
Differential Revision: http://reviews.llvm.org/D6589
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223915 91177308-0d34-0410-b5e6-96231b3b80d8
--xunit-xml-output saves test results to disk in JUnit's xml format. This will allow Jenkins to report the details of a lit run.
Based on a patch by David Chisnall.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223163 91177308-0d34-0410-b5e6-96231b3b80d8
Currently they are passed to tests of llvm itself, but not, for example, lld.
With this patch the options are visible in every test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221198 91177308-0d34-0410-b5e6-96231b3b80d8
execution of a shell command. This can happen for example if the
``RUN:`` line calls a python script which can work correctly under
Linux/OSX but will not work under Windows. A more useful error message
is now shown rather than an unhelpful backtrace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220227 91177308-0d34-0410-b5e6-96231b3b80d8
As far as I can tell UTF-8 has been supported since the beginning of Python's
codec support, and it's the de facto standard for text these days, at least
for primarily-English text. This allows us to put Unicode into lit RUN lines.
rdar://problem/18311663
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217688 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch changes the way xfail and unsupported tests are displayed.
This output is only displayed when the --show-unsupported/--show-xfail flags are passed to lit.
Currently xfail/unsupported tests are printed during the run of the test-suite. I think its better to display this information during the summary instead.
This patch removes the printing of these tests from when they are run to the summary.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D4842
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215809 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch add a --show-xfail flag. If this flag is specified then each xfail test will be printed to output.
When it is not given xfail tests are ignored. Ignoring xfail tests is the current behavior.
This flag is meant to mirror the --show-unsupported flag that was recently added.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D4750
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214609 91177308-0d34-0410-b5e6-96231b3b80d8
Add all inputs to the array, except those starting with @, which
are treated as response files and expanded.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211119 91177308-0d34-0410-b5e6-96231b3b80d8
It was pointed out that this breaks the "virtual test discovery"
mechanism, which allows for narming tests in the test exec root.
Reverting until I can figure out how to fix this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211048 91177308-0d34-0410-b5e6-96231b3b80d8
It would previously say things like
warning: input 'test/Frontend/foo.c' contained no tests
and have the user pull their hair trying to figure out what's wrong with that
file. This patch changes the message to the much clearer:
warning: no such file or directory: 'test/Frontend/foo.c'
Differential Revision: http://reviews.llvm.org/D4097
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210597 91177308-0d34-0410-b5e6-96231b3b80d8
Clang's lit cfg already detects the currently selected SDK via
"xcrun --show-sdk-path". The same thing should be done for compiler-rt tests,
to make them work on recent OS X versions. Instead of duplicating the detection
code, this patch extracts the detection function into a lit.util method.
Patch by Kuba Brecka (kuba.brecka@gmail.com),
reviewed at http://reviews.llvm.org/D4072
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210534 91177308-0d34-0410-b5e6-96231b3b80d8
This commit prospectively brings the benefits of r198766 to older supported
Python versions (2.5+).
Tested with Python 2.6, 2.7, 3.1 and 3.3 (!)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199009 91177308-0d34-0410-b5e6-96231b3b80d8
On the other hand, exec(compile()) doesn't work in older Python versions in the
2.x series.
This commit introduces exec(compile()) with a fallback to plain exec(). That'll
hopefully hit the sweet spot in terms of version support.
Followup to r198766 which added enhanced source locations for lit cfg parsing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199006 91177308-0d34-0410-b5e6-96231b3b80d8
Python doesn't do a good job at diagnosing string exec() so use execfile()
where available.
This should be a timesaver when trying to get to the bottom of build bot
failures.
Before:
File "llvm/utils/lit/lit/TestingConfig.py", line 93, in load_from_path
exec("exec data in cfg_globals")
File "<string>", line 1, in <module>
File "<string>", line 194, in <module>
NameError: name 'typo' is not defined
After:
File "llvm/utils/lit/lit/TestingConfig.py", line 95, in load_from_path
execfile(path, cfg_globals)
File "clang/test/lit.cfg", line 194, in <module>
typo
^~~~
NameError: name 'typo' is not defined
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198766 91177308-0d34-0410-b5e6-96231b3b80d8
Remove the stat call error reporting for now.
It wasn't essential so silent fallback should be fine here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198155 91177308-0d34-0410-b5e6-96231b3b80d8
