Summary: I think this is probably a bug, but I'm putting this up for review just to be sure. I think that `lit.util.capture` should decode the resulting string in the same way `lit.util.executeCommand` does.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D6769
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@225681 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
The following types can be encoded and decoded by the json library:
`dict`, `list`, `tuple`, `str`, `unicode`, `int`, `long`, `float`, `bool`, `NoneType`.
`JSONMetricValue` can be constructed with any of these types, and used as part of Test.Result.
This patch also adds a toMetricValue function that converts a value into a MetricValue.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits, llvm-commits
Differential Revision: http://reviews.llvm.org/D6576
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224628 91177308-0d34-0410-b5e6-96231b3b80d8
We were already requiring 2.5, which meant that people on old linux distros
had to upgrade anyway.
Requiring python 2.6 will make supporting 3.X easier as we can use the 3.X
exception syntax.
According to the discussion on llvmdev, there is not much value is requiring
just 2.6, we may as well just require 2.7.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224129 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch gives me just enough to leverage the existing functionality in `TestRunner` for use in `libc++` and `libc++abi` .
It does the following:
* Adds the `UNSUPPORTED` tag to `TestRunner.parseIntegratedTestScript`.
* Allows `parseIntegratedTestScript` to return an empty script if a script is not required by the caller.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: cfe-commits, llvm-commits
Differential Revision: http://reviews.llvm.org/D6589
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223915 91177308-0d34-0410-b5e6-96231b3b80d8
--xunit-xml-output saves test results to disk in JUnit's xml format. This will allow Jenkins to report the details of a lit run.
Based on a patch by David Chisnall.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223163 91177308-0d34-0410-b5e6-96231b3b80d8
Currently they are passed to tests of llvm itself, but not, for example, lld.
With this patch the options are visible in every test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@221198 91177308-0d34-0410-b5e6-96231b3b80d8
execution of a shell command. This can happen for example if the
``RUN:`` line calls a python script which can work correctly under
Linux/OSX but will not work under Windows. A more useful error message
is now shown rather than an unhelpful backtrace.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@220227 91177308-0d34-0410-b5e6-96231b3b80d8
As far as I can tell UTF-8 has been supported since the beginning of Python's
codec support, and it's the de facto standard for text these days, at least
for primarily-English text. This allows us to put Unicode into lit RUN lines.
rdar://problem/18311663
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@217688 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch changes the way xfail and unsupported tests are displayed.
This output is only displayed when the --show-unsupported/--show-xfail flags are passed to lit.
Currently xfail/unsupported tests are printed during the run of the test-suite. I think its better to display this information during the summary instead.
This patch removes the printing of these tests from when they are run to the summary.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D4842
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@215809 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This patch add a --show-xfail flag. If this flag is specified then each xfail test will be printed to output.
When it is not given xfail tests are ignored. Ignoring xfail tests is the current behavior.
This flag is meant to mirror the --show-unsupported flag that was recently added.
Reviewers: ddunbar, EricWF
Reviewed By: EricWF
Subscribers: llvm-commits
Differential Revision: http://reviews.llvm.org/D4750
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@214609 91177308-0d34-0410-b5e6-96231b3b80d8
Add all inputs to the array, except those starting with @, which
are treated as response files and expanded.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211119 91177308-0d34-0410-b5e6-96231b3b80d8
It was pointed out that this breaks the "virtual test discovery"
mechanism, which allows for narming tests in the test exec root.
Reverting until I can figure out how to fix this.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@211048 91177308-0d34-0410-b5e6-96231b3b80d8
It would previously say things like
warning: input 'test/Frontend/foo.c' contained no tests
and have the user pull their hair trying to figure out what's wrong with that
file. This patch changes the message to the much clearer:
warning: no such file or directory: 'test/Frontend/foo.c'
Differential Revision: http://reviews.llvm.org/D4097
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210597 91177308-0d34-0410-b5e6-96231b3b80d8
Clang's lit cfg already detects the currently selected SDK via
"xcrun --show-sdk-path". The same thing should be done for compiler-rt tests,
to make them work on recent OS X versions. Instead of duplicating the detection
code, this patch extracts the detection function into a lit.util method.
Patch by Kuba Brecka (kuba.brecka@gmail.com),
reviewed at http://reviews.llvm.org/D4072
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@210534 91177308-0d34-0410-b5e6-96231b3b80d8
This commit prospectively brings the benefits of r198766 to older supported
Python versions (2.5+).
Tested with Python 2.6, 2.7, 3.1 and 3.3 (!)
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199009 91177308-0d34-0410-b5e6-96231b3b80d8
On the other hand, exec(compile()) doesn't work in older Python versions in the
2.x series.
This commit introduces exec(compile()) with a fallback to plain exec(). That'll
hopefully hit the sweet spot in terms of version support.
Followup to r198766 which added enhanced source locations for lit cfg parsing.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@199006 91177308-0d34-0410-b5e6-96231b3b80d8
Python doesn't do a good job at diagnosing string exec() so use execfile()
where available.
This should be a timesaver when trying to get to the bottom of build bot
failures.
Before:
File "llvm/utils/lit/lit/TestingConfig.py", line 93, in load_from_path
exec("exec data in cfg_globals")
File "<string>", line 1, in <module>
File "<string>", line 194, in <module>
NameError: name 'typo' is not defined
After:
File "llvm/utils/lit/lit/TestingConfig.py", line 95, in load_from_path
execfile(path, cfg_globals)
File "clang/test/lit.cfg", line 194, in <module>
typo
^~~~
NameError: name 'typo' is not defined
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198766 91177308-0d34-0410-b5e6-96231b3b80d8
Remove the stat call error reporting for now.
It wasn't essential so silent fallback should be fine here.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198155 91177308-0d34-0410-b5e6-96231b3b80d8
Add option -i to prioritize test runs by source file modification time and
previous failure state.
This optimal scheduling reduces typical test-and-fix iteration times to a
matter of seconds by rapidly answering the questions:
1) Did my recent change fix tests that were previously failing?
2) Do the tests I just wrote / modified still work?
The current implementation requires write permissions to the source tree
because it uses mtimes to track failures.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@198150 91177308-0d34-0410-b5e6-96231b3b80d8
The error raised by Python varies by platform(!), so let's just catch any
exception and fall back.
Thanks to Sylvestre Ledru for noticing this on a Debian / Python 2.7 system
running code coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193516 91177308-0d34-0410-b5e6-96231b3b80d8
so try PATH next. Assume it is sane enough to cover the usual system
bash locations too, but the old list is not good enough for NetBSD.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193471 91177308-0d34-0410-b5e6-96231b3b80d8
If multiprocessing was requested, detected as available and subsequently failed
to initialize it's worth letting the user know about it before falling back to
threads.
This condition can arise in certain OpenBSD / FreeBSD Python versions.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@193465 91177308-0d34-0410-b5e6-96231b3b80d8
