Commit Graph

30 Commits

Author SHA1 Message Date
2111b5758f [LIT] Fix failing LIT tests
Summary:
I spend some time trying to get the LIT test suite passing. Here are the changes that I needed to make on my machine.

I made the following changes for the following reasons.

1. google-test.py: The Google test format now checks for "[  PASSED  ] 1 test." to check if a test passes.
2. discovery.py: The output appears in a different order on my machine than it did in the test.
3. unittest-adaptor.py: The output appears in a different order on my machine than it did in the test.
4. The classname is now formed differently in `getJUnitXML(...)`.

I'm not sure what is causing the output order to differ in discovery.py and unittest-adaptor.py. Does anybody have any thoughts?

Reviewers: ddunbar, danalbert, jroelofs

Reviewed By: jroelofs

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D9864

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@239663 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-13 06:55:44 +00:00
708bc15ff0 Require python 2.7.
We were already requiring 2.5, which meant that people on old linux distros
had to upgrade anyway.

Requiring python 2.6 will make supporting 3.X easier as we can use the 3.X
exception syntax.

According to the discussion on llvmdev, there is not much value is requiring
just 2.6, we may as well just require 2.7.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@224129 91177308-0d34-0410-b5e6-96231b3b80d8
2014-12-12 15:29:31 +00:00
0baca9564c Add a test-case for lit xunit output
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@223307 91177308-0d34-0410-b5e6-96231b3b80d8
2014-12-03 23:01:35 +00:00
2849503ab2 [lit] Add an --output option, for writing results in a machine readable form.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190738 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-14 01:19:17 +00:00
ff058f0a70 [lit] Add support for attach arbitrary metrics to test results.
- This is a work-in-progress and all details are subject to change, but I am
   trying to build up support for allowing lit to be used as a driver for
   performance tests (or other tests which might want to record information
   beyond simple PASS/FAIL).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@190535 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-11 17:45:11 +00:00
4b3e5f24ec [lit] Allow config files to pass arbitrary values to child configs.
- This aligns with how existing test suites end up wanting to use the local
   config files, conceptually it makes sense to consider them to be inherited.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189885 91177308-0d34-0410-b5e6-96231b3b80d8
2013-09-03 23:32:55 +00:00
e0ff96c112 [lit] [tests] Add missing test input file.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189561 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-29 03:02:34 +00:00
cb4bfb3409 [lit] Fix internal shell's argv[0] handling.
- At least on OS X, it is important for correct behavior of /bin/[ that argv[0]
   is passed as written, and not as the full executable path.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@189559 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-29 02:52:10 +00:00
f72bc79226 [lit] Lift XFAIL handling to core infrastructure.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188949 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-21 22:26:42 +00:00
4fa5939268 [lit] Add test coverage of gtest format.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188417 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 22:21:11 +00:00
80a06baec2 [lit] Ensure test output is converted to strings where possible.
- This cleans up the text output of failing tests when run under PY3.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188416 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 22:21:01 +00:00
35d5e9044c [lit] Support parsing scripts with inconsistent or invalid encodings.
- For whatever reason, we have a lot of test files with bogus unicode
   characters. This patch allows those scripts to still be parsed on Python3 by
   changing the parsing logic to work on binary files, and only require the
   actual script commands to be convertible to ascii.

 - This patch has been tweaked to now ensure that the command strings are not of
   unicode type on Python 2.6-7.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188398 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 18:22:41 +00:00
14a5c695a9 Revert r188376, "[lit] Support parsing scripts with inconsistent or invalid encodings.", this doesn't work yet for bots using the internal shell.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188379 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 15:55:25 +00:00
47e7c040f4 [lit] Support parsing scripts with inconsistent or invalid encodings.
- For whatever reason, we have a lot of test files with bogus unicode
   characters. This patch allows those scripts to still be parsed on Python3 by
   changing the parsing logic to work on binary files, and only require the
   actual script commands to be convertible to ascii.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188376 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 15:24:58 +00:00
6b78ef3762 [lit] Fix tests to execute lit with same python as invoked with.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188358 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-14 05:07:09 +00:00
4981a6ca97 [lit] Update lit's own tests to use lit_config and lit package, as appropriate.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188107 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-09 21:39:17 +00:00
abb9de5257 [lit] Change --show-{tests,suites} to exit after printing.
- This is a more sensible behavior than printing and also running tests.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@188009 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-08 20:59:25 +00:00
6bcb407d90 [lit] tests: Change discovery test to use --show-tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@187851 91177308-0d34-0410-b5e6-96231b3b80d8
2013-08-07 03:12:51 +00:00
406ef44707 lit: Fix infinite recursion when an out-of-tree test root is located inside the source test root.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179402 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-12 19:09:09 +00:00
f2c2b20090 lit: Add a test for discovery w/ test_exec_root (out-of-tree test root).
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179401 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-12 19:08:57 +00:00
6be0ae59ec lit: Add a test for discovery when exact test names are given.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179247 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-11 00:31:22 +00:00
2b570ce5aa lit: Add a trivial test of the basic progress bar.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@179243 91177308-0d34-0410-b5e6-96231b3b80d8
2013-04-11 00:05:37 +00:00
ac1b2e5550 [lit] Add a test for internal shell execution behaviors.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174102 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 22:15:20 +00:00
86aa95fc28 [lit] Add a test for the various ShTest format features.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174072 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 18:05:01 +00:00
47896090f3 [lit] Move unittest adaptor code into discovery module.
- Also, add a test for it.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174019 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 01:23:39 +00:00
c9fd0a57f7 [lit] Run the ShUtil tests as part of the test suite.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174018 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 01:23:26 +00:00
84d3c61c03 [lit] Set parallel mode for coverage tests.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174017 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 01:23:18 +00:00
ada2de95ac [lit] Add a basic discovery test.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174001 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 00:21:51 +00:00
412d805841 [lit] Add a script for checking test coverage.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@174000 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 00:21:44 +00:00
356dcac20e [lit] Add a test suite for lit itself.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@173999 91177308-0d34-0410-b5e6-96231b3b80d8
2013-01-31 00:21:39 +00:00