[Support] Beef up and expose the response file parsing in llvm::cl

The plan is to use it for clang and lld.

Major behavior changes:
- We can now parse UTF-16 files that have a byte order mark.
- PR16209: Don't drop backslashes on the floor if they don't escape
  anything.

The actual parsing loop was based on code from Clang's driver.cpp,
although it's been rewritten to track its state with control flow rather
than state variables.

Reviewers: hans

Differential Revision: http://llvm-reviews.chandlerc.com/D1170

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@186587 91177308-0d34-0410-b5e6-96231b3b80d8

test/Other/ResponseFile.ll

@@ -1,5 +1,9 @@
-; RUN: echo %s > %t.list
-; RUN: llvm-as @%t.list -o %t.bc
+; Test that we can recurse, at least a little bit.  The -time-passes flag here
+; is a hack to make sure that neither echo nor the shell expands the response
+; file for us.  Tokenization with quotes is tested in unittests.
+; RUN: echo %s > %t.list1
+; RUN: echo "-time-passes @%t.list1" > %t.list2
+; RUN: llvm-as @%t.list2 -o %t.bc
 ; RUN: llvm-nm %t.bc 2>&1 | FileCheck %s
 
 ; CHECK: T foobar