Implement and document the llvm.eh.resume intrinsic, which is

transformed by the inliner into a branch to the enclosing landing pad
(when inlined through an invoke).  If not so optimized, it is lowered
DWARF EH preparation into a call to _Unwind_Resume (or _Unwind_SjLj_Resume
as appropriate).  Its chief advantage is that it takes both the
exception value and the selector value as arguments, meaning that there
is zero effort in recovering these;  however, the frontend is required
to pass these down, which is not actually particularly difficult.

Also document the behavior of landing pads a bit better, and make it
clearer that it's okay that personality functions don't always land at
landing pads.  This is just a fact of life.  Don't write optimizations that
rely on pushing things over an unwind edge.



git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@132253 91177308-0d34-0410-b5e6-96231b3b80d8

docs/ExceptionHandling.html

@@ -35,6 +35,7 @@
   <ol>
   	<li><a href="#llvm_eh_exception"><tt>llvm.eh.exception</tt></a></li>
   	<li><a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a></li>
+  	<li><a href="#llvm_eh_resume"><tt>llvm.eh.resume</tt></a></li>
   	<li><a href="#llvm_eh_typeid_for"><tt>llvm.eh.typeid.for</tt></a></li>
   	<li><a href="#llvm_eh_sjlj_setjmp"><tt>llvm.eh.sjlj.setjmp</tt></a></li>
   	<li><a href="#llvm_eh_sjlj_longjmp"><tt>llvm.eh.sjlj.longjmp</tt></a></li>
@@ -317,15 +318,28 @@
 
 <div>
 
-<p>To handle destructors and cleanups in <tt>try</tt> code, control may not run
+<p>A cleanup is extra code which needs to be run as part of unwinding
-   directly from a landing pad to the first catch.  Control may actually flow
+   a scope.  C++ destructors are a prominent example, but other
-   from the landing pad to clean up code and then to the first catch.  Since the
+   languages and language extensions provide a variety of different
-   required clean up for each <tt>invoke</tt> in a <tt>try</tt> may be different
+   kinds of cleanup.  In general, a landing pad may need to run
-   (e.g. intervening constructor), there may be several landing pads for a given
+   arbitrary amounts of cleanup code before actually entering a catch
-   try.  If cleanups need to be run, an <tt>i32 0</tt> should be passed as the
+   block.  To indicate the presence of cleanups, a landing pad's call
-   last <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> argument.
+   to <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> should
-   However, when using DWARF exception handling with C++, a <tt>i8* null</tt>
+   end with the argument <tt>i32 0</tt>; otherwise, the unwinder will
-   <a href="#restrictions">must</a> be passed instead.</p>
+   not stop at the landing pad if there are no catches or filters that
+   require it to.</p>
+
+<p>Do not allow a new exception to propagate out of the execution of a
+   cleanup.  This can corrupt the internal state of the unwinder.
+   Different languages describe different high-level semantics for
+   these situations: for example, C++ requires that the process be
+   terminated, whereas Ada cancels both exceptions and throws a third.</p>
+
+<p>When all cleanups have completed, if the exception is not handled
+   by the current function, resume unwinding by calling the
+   <a href="#llvm_eh_resume"><tt>llvm.eh.resume</tt></a> intrinsic,
+   passing in the results of <tt>llvm.eh.exception</tt> and
+   <tt>llvm.eh.selector</tt> for the original landing pad.</p>
 
 </div>
 
@@ -363,22 +377,29 @@
 
 <div>
 
-<p>The semantics of the invoke instruction require that any exception that
+<p>The unwinder delegates the decision of whether to stop in a call
-   unwinds through an invoke call should result in a branch to the invoke's
+   frame to that call frame's language-specific personality function.
-   unwind label.  However such a branch will only happen if the
+   Not all personalities functions guarantee that they will stop to
-   <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> matches. Thus in
+   perform cleanups: for example, the GNU C++ personality doesn't do
-   order to ensure correct operation, the front-end must only generate
+   so unless the exception is actually caught somewhere further up the
-   <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> calls that are
+   stack.  When using this personality to implement EH for a language
-   guaranteed to always match whatever exception unwinds through the invoke.
+   that guarantees that cleanups will always be run, be sure to
-   For most languages it is enough to pass zero, indicating the presence of
+   indicate a catch-all in the
-   a <a href="#cleanups">cleanup</a>, as the
+   <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> call
-   last <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> argument.
+   rather than just cleanups.</p>
-   However for C++ this is not sufficient, because the C++ personality function
+
-   will terminate the program if it detects that unwinding the exception only
+<p>In order for inlining to behave correctly, landing pads must be
-   results in matches with cleanups.  For C++ a <tt>null i8*</tt> should be
+   prepared to handle selector results that they did not originally
-   passed as the last <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a>
+   advertise.  Suppose that a function catches exceptions of
-   argument instead.  This is interpreted as a catch-all by the C++ personality
+   type <tt>A</tt>, and it's inlined into a function that catches
-   function, and will always match.</p>
+   exceptions of type <tt>B</tt>.  The inliner will update the
+   selector for the inlined landing pad to include the fact
+   that <tt>B</tt> is caught.  If that landing pad assumes that it
+   will only be entered to catch an <tt>A</tt>, it's in for a rude
+   surprise.  Consequently, landing pads must test for the selector
+   results they understand and then resume exception propagation
+   with the <a href="#llvm_eh_resume"><tt>llvm.eh.resume</tt></a>
+   intrinsic if none of the conditions match.</p>
 
 </div>
 
@@ -424,22 +445,32 @@
 <p>This intrinsic is used to compare the exception with the given type infos,
    filters and cleanups.</p>
 
-<p><a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> takes a minimum of
+<p><a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> takes a
-   three arguments.  The first argument is the reference to the exception
+   minimum of three arguments.  The first argument is the reference to
-   structure. The second argument is a reference to the personality function to
+   the exception structure. The second argument is a reference to the
-   be used for this try catch sequence. Each of the remaining arguments is
+   personality function to be used for this try catch sequence. Each
-   either a reference to the type info for a catch statement,
+   of the remaining arguments is either a reference to the type info
-   a <a href="#throw_filters">filter</a> expression, or the number zero
+   for a catch statement, a <a href="#throw_filters">filter</a>
-   representing a <a href="#cleanups">cleanup</a>.  The exception is tested
+   expression, or the number zero representing
-   against the arguments sequentially from first to last.  The result of
+   a <a href="#cleanups">cleanup</a>.  The exception is tested against
-   the <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> is a positive
+   the arguments sequentially from first to last.  The result of
-   number if the exception matched a type info, a negative number if it matched
+   the <a href="#llvm_eh_selector"><tt>llvm.eh.selector</tt></a> is a
-   a filter, and zero if it matched a cleanup.  If nothing is matched, the
+   positive number if the exception matched a type info, a negative
-   behaviour of the program is <a href="#restrictions">undefined</a>.  If a type
+   number if it matched a filter, and zero if it matched a cleanup.
-   info matched then the selector value is the index of the type info in the
+   If nothing is matched, or if only a cleanup is matched, different
-   exception table, which can be obtained using the
+   personality functions may or may not cause control to stop at the
+   landing pad; see <a href="#restrictions">the restrictions</a> for
+   more information.  If a type info matched then the selector value
+   is the index of the type info in the exception table, which can be
+   obtained using the
    <a href="#llvm_eh_typeid_for"><tt>llvm.eh.typeid.for</tt></a> intrinsic.</p>
 
+<p>If a landing pad containing a call to <tt>llvm.eh.selector</tt> is
+   inlined into an <tt>invoke</tt> instruction, the selector arguments
+   for the outer landing pad are appended to those of the inlined
+   landing pad.  Consequently, landing pads must be written to ignore
+   selector values that they did not originally advertise.</p>
+
 </div>
 
 <!-- ======================================================================= -->
@@ -460,6 +491,33 @@
 
 </div>
 
+<!-- ======================================================================= -->
+<h4>
+  <a name="llvm_eh_resume">llvm.eh.resume</a>
+</h4>
+
+<div>
+
+<pre>
+  void %<a href="#llvm_eh_resume">llvm.eh.resume</a>(i8*, i32) noreturn
+</pre>
+
+<p>This intrinsic is used to resume propagation of an exception after
+   landing at a landing pad.  The first argument should be the result
+   of <a href="#llvm_eh_exception">llvm.eh.exception</a> for that
+   landing pad, and the second argument should be the result of
+   <a href="#llvm_eh_selector">llvm.eh.selector</a>.  When a call to
+   this intrinsic is inlined into an invoke, the call is transformed
+   into a branch to the invoke's unwind destination, using its
+   arguments in place of the calls
+   to <a href="#llvm_eh_exception">llvm.eh.exception</a> and
+   <a href="#llvm_eh_selector">llvm.eh.selector</a> there.</p>
+
+<p>This intrinsic is not implicitly <tt>nounwind</tt>; calls to it
+   will always throw.  It may not be invoked.</p>
+
+</div>
+
 <!-- ======================================================================= -->
 <h4>
   <a name="llvm_eh_sjlj_setjmp">llvm.eh.sjlj.setjmp</a>

lib/CodeGen/DwarfEHPrepare.cpp

@@ -30,6 +30,7 @@ using namespace llvm;
 
 STATISTIC(NumLandingPadsSplit,     "Number of landing pads split");
 STATISTIC(NumUnwindsLowered,       "Number of unwind instructions lowered");
+STATISTIC(NumResumesLowered,       "Number of eh.resume calls lowered");
 STATISTIC(NumExceptionValuesMoved, "Number of eh.exception calls moved");
 
 namespace {
@@ -63,7 +64,7 @@ namespace {
     BBSet LandingPads;
 
     bool NormalizeLandingPads();
-    bool LowerUnwinds();
+    bool LowerUnwindsAndResumes();
     bool MoveExceptionValueCalls();
 
     Instruction *CreateExceptionValueCall(BasicBlock *BB);
@@ -480,20 +481,25 @@ bool DwarfEHPrepare::NormalizeLandingPads() {
 /// rethrowing any previously caught exception.  This will crash horribly
 /// at runtime if there is no such exception: using unwind to throw a new
 /// exception is currently not supported.
-bool DwarfEHPrepare::LowerUnwinds() {
+bool DwarfEHPrepare::LowerUnwindsAndResumes() {
-  SmallVector<TerminatorInst*, 16> UnwindInsts;
+  SmallVector<Instruction*, 16> ResumeInsts;
 
-  for (Function::iterator I = F->begin(), E = F->end(); I != E; ++I) {
+  for (Function::iterator fi = F->begin(), fe = F->end(); fi != fe; ++fi) {
-    TerminatorInst *TI = I->getTerminator();
+    for (BasicBlock::iterator bi = fi->begin(), be = fi->end(); bi != be; ++bi){
-    if (isa<UnwindInst>(TI))
+      if (isa<UnwindInst>(bi))
-      UnwindInsts.push_back(TI);
+        ResumeInsts.push_back(bi);
+      else if (CallInst *call = dyn_cast<CallInst>(bi))
+        if (Function *fn = dyn_cast<Function>(call->getCalledValue()))
+          if (fn->getName() == "llvm.eh.resume")
+            ResumeInsts.push_back(bi);
+    }
   }
 
-  if (UnwindInsts.empty()) return false;
+  if (ResumeInsts.empty()) return false;
 
   // Find the rewind function if we didn't already.
   if (!RewindFunction) {
-    LLVMContext &Ctx = UnwindInsts[0]->getContext();
+    LLVMContext &Ctx = ResumeInsts[0]->getContext();
     std::vector<const Type*>
       Params(1, Type::getInt8PtrTy(Ctx));
     FunctionType *FTy = FunctionType::get(Type::getVoidTy(Ctx),
@@ -504,24 +510,35 @@ bool DwarfEHPrepare::LowerUnwinds() {
 
   bool Changed = false;
 
-  for (SmallVectorImpl<TerminatorInst*>::iterator
+  for (SmallVectorImpl<Instruction*>::iterator
-         I = UnwindInsts.begin(), E = UnwindInsts.end(); I != E; ++I) {
+         I = ResumeInsts.begin(), E = ResumeInsts.end(); I != E; ++I) {
-    TerminatorInst *TI = *I;
+    Instruction *RI = *I;
 
-    // Replace the unwind instruction with a call to _Unwind_Resume (or the
+    // Replace the resuming instruction with a call to _Unwind_Resume (or the
-    // appropriate target equivalent) followed by an UnreachableInst.
+    // appropriate target equivalent).
+
+    llvm::Value *ExnValue;
+    if (isa<UnwindInst>(RI))
+      ExnValue = CreateExceptionValueCall(RI->getParent());
+    else
+      ExnValue = cast<CallInst>(RI)->getArgOperand(0);
 
     // Create the call...
-    CallInst *CI = CallInst::Create(RewindFunction,
+    CallInst *CI = CallInst::Create(RewindFunction, ExnValue, "", RI);
-                                    CreateExceptionValueCall(TI->getParent()),
-                                    "", TI);
     CI->setCallingConv(TLI->getLibcallCallingConv(RTLIB::UNWIND_RESUME));
-    // ...followed by an UnreachableInst.
-    new UnreachableInst(TI->getContext(), TI);
 
-    // Nuke the unwind instruction.
+    // ...followed by an UnreachableInst, if it was an unwind.
-    TI->eraseFromParent();
+    // Calls to llvm.eh.resume are typically already followed by this.
-    ++NumUnwindsLowered;
+    if (isa<UnwindInst>(RI))
+      new UnreachableInst(RI->getContext(), RI);
+
+    // Nuke the resume instruction.
+    RI->eraseFromParent();
+
+    if (isa<UnwindInst>(RI))
+      ++NumUnwindsLowered;
+    else
+      ++NumResumesLowered;
     Changed = true;
   }
 
@@ -657,8 +674,8 @@ bool DwarfEHPrepare::runOnFunction(Function &Fn) {
   // basic block where an invoke unwind edge ends).
   Changed |= NormalizeLandingPads();
 
-  // Turn unwind instructions into libcalls.
+  // Turn unwind instructions and eh.resume calls into libcalls.
-  Changed |= LowerUnwinds();
+  Changed |= LowerUnwindsAndResumes();
 
   // TODO: Move eh.selector calls to landing pads and combine them.
 

lib/Transforms/Utils/InlineFunction.cpp

@@ -45,66 +45,192 @@ bool llvm::InlineFunction(InvokeInst *II, InlineFunctionInfo &IFI) {
   return InlineFunction(CallSite(II), IFI);
 }
 
+/// [LIBUNWIND] Find the (possibly absent) call to @llvm.eh.selector in
+/// the given landing pad.
+static EHSelectorInst *findSelectorForLandingPad(BasicBlock *lpad) {
+  // The llvm.eh.exception call is required to be in the landing pad.
+  for (BasicBlock::iterator i = lpad->begin(), e = lpad->end(); i != e; i++) {
+    EHExceptionInst *exn = dyn_cast<EHExceptionInst>(i);
+    if (!exn) continue;
+
+    EHSelectorInst *selector = 0;
+    for (Instruction::use_iterator
+           ui = exn->use_begin(), ue = exn->use_end(); ui != ue; ++ui) {
+      EHSelectorInst *sel = dyn_cast<EHSelectorInst>(*ui);
+      if (!sel) continue;
+
+      // Immediately accept an eh.selector in the landing pad.
+      if (sel->getParent() == lpad) return sel;
+
+      // Otherwise, use the first selector we see.
+      if (!selector) selector = sel;
+    }
+
+    return selector;
+  }
+
+  return 0;
+}
+
 namespace {
   /// A class for recording information about inlining through an invoke.
   class InvokeInliningInfo {
-    BasicBlock *UnwindDest;
+    BasicBlock *OuterUnwindDest;
+    EHSelectorInst *OuterSelector;
+    BasicBlock *InnerUnwindDest;
+    PHINode *InnerExceptionPHI;
+    PHINode *InnerSelectorPHI;
     SmallVector<Value*, 8> UnwindDestPHIValues;
 
   public:
-    InvokeInliningInfo(InvokeInst *II) : UnwindDest(II->getUnwindDest()) {
+    InvokeInliningInfo(InvokeInst *II) :
+      OuterUnwindDest(II->getUnwindDest()), OuterSelector(0),
+      InnerUnwindDest(0), InnerExceptionPHI(0), InnerSelectorPHI(0) {
+
       // If there are PHI nodes in the unwind destination block, we
       // need to keep track of which values came into them from the
       // invoke before removing the edge from this block.
-      llvm::BasicBlock *InvokeBlock = II->getParent();
+      llvm::BasicBlock *invokeBB = II->getParent();
-      for (BasicBlock::iterator I = UnwindDest->begin(); isa<PHINode>(I); ++I) {
+      for (BasicBlock::iterator I = OuterUnwindDest->begin();
-        PHINode *PN = cast<PHINode>(I);
+             isa<PHINode>(I); ++I) {
         // Save the value to use for this edge.
-        llvm::Value *Incoming = PN->getIncomingValueForBlock(InvokeBlock);
+        PHINode *phi = cast<PHINode>(I);
-        UnwindDestPHIValues.push_back(Incoming);
+        UnwindDestPHIValues.push_back(phi->getIncomingValueForBlock(invokeBB));
       }
     }
 
-    BasicBlock *getUnwindDest() const {
+    /// The outer unwind destination is the target of unwind edges
-      return UnwindDest;
+    /// introduced for calls within the inlined function.
+    BasicBlock *getOuterUnwindDest() const {
+      return OuterUnwindDest;
     }
 
+    EHSelectorInst *getOuterSelector() {
+      if (!OuterSelector)
+        OuterSelector = findSelectorForLandingPad(OuterUnwindDest);
+      return OuterSelector;
+    }
+
+    BasicBlock *getInnerUnwindDest();
+
+    bool forwardEHResume(CallInst *call, BasicBlock *src);
+
     /// Add incoming-PHI values to the unwind destination block for
     /// the given basic block, using the values for the original
     /// invoke's source block.
     void addIncomingPHIValuesFor(BasicBlock *BB) const {
-      BasicBlock::iterator I = UnwindDest->begin();
+      addIncomingPHIValuesForInto(BB, OuterUnwindDest);
+    }
+
+    void addIncomingPHIValuesForInto(BasicBlock *src, BasicBlock *dest) const {
+      BasicBlock::iterator I = dest->begin();
       for (unsigned i = 0, e = UnwindDestPHIValues.size(); i != e; ++i, ++I) {
-        PHINode *PN = cast<PHINode>(I);
+        PHINode *phi = cast<PHINode>(I);
-        PN->addIncoming(UnwindDestPHIValues[i], BB);
+        phi->addIncoming(UnwindDestPHIValues[i], src);
       }
     }
   };
 }
 
-/// [LIBUNWIND] Check whether the given value is the _Unwind_Resume
+/// Replace all the instruction uses of a value with a different value.
-/// function specified by the Itanium EH ABI.
+/// This has the advantage of not screwing up the CallGraph.
-static bool isUnwindResume(Value *value) {
+static void replaceAllInsnUsesWith(Instruction *insn, Value *replacement) {
-  Function *fn = dyn_cast<Function>(value);
+  for (Value::use_iterator i = insn->use_begin(), e = insn->use_end();
-  if (!fn) return false;
+       i != e; ) {
-
+    Use &use = i.getUse();
-  // declare void @_Unwind_Resume(i8*)
+    ++i;
-  if (fn->getName() != "_Unwind_Resume") return false;
+    if (isa<Instruction>(use.getUser()))
-  const FunctionType *fnType = fn->getFunctionType();
+      use.set(replacement);
-  if (!fnType->getReturnType()->isVoidTy()) return false;
+  }
-  if (fnType->isVarArg()) return false;
-  if (fnType->getNumParams() != 1) return false;
-  const PointerType *paramType = dyn_cast<PointerType>(fnType->getParamType(0));
-  return (paramType && paramType->getElementType()->isIntegerTy(8));
 }
 
-/// [LIBUNWIND] Find the (possibly absent) call to @llvm.eh.selector in
+/// Get or create a target for the branch out of rewritten calls to
-/// the given landing pad.
+/// llvm.eh.resume.
-static EHSelectorInst *findSelectorForLandingPad(BasicBlock *lpad) {
+BasicBlock *InvokeInliningInfo::getInnerUnwindDest() {
-  for (BasicBlock::iterator i = lpad->begin(), e = lpad->end(); i != e; i++)
+  if (InnerUnwindDest) return InnerUnwindDest;
-    if (EHSelectorInst *selector = dyn_cast<EHSelectorInst>(i))
+
-      return selector;
+  // Find and hoist the llvm.eh.exception and llvm.eh.selector calls
-  return 0;
+  // in the outer landing pad to immediately following the phis.
+  EHSelectorInst *selector = getOuterSelector();
+  if (!selector) return 0;
+
+  // The call to llvm.eh.exception *must* be in the landing pad.
+  Instruction *exn = cast<Instruction>(selector->getArgOperand(0));
+  assert(exn->getParent() == OuterUnwindDest);
+
+  // TODO: recognize when we've already done this, so that we don't
+  // get a linear number of these when inlining calls into lots of
+  // invokes with the same landing pad.
+
+  // Do the hoisting.
+  Instruction *splitPoint = exn->getParent()->getFirstNonPHI();
+  assert(splitPoint != selector && "selector-on-exception dominance broken!");
+  if (splitPoint == exn) {
+    selector->removeFromParent();
+    selector->insertAfter(exn);
+    splitPoint = selector->getNextNode();
+  } else {
+    exn->moveBefore(splitPoint);
+    selector->moveBefore(splitPoint);
+  }
+
+  // Split the landing pad.
+  InnerUnwindDest = OuterUnwindDest->splitBasicBlock(splitPoint,
+                                        OuterUnwindDest->getName() + ".body");
+
+  // The number of incoming edges we expect to the inner landing pad.
+  const unsigned phiCapacity = 2;
+
+  // Create corresponding new phis for all the phis in the outer landing pad.
+  BasicBlock::iterator insertPoint = InnerUnwindDest->begin();
+  BasicBlock::iterator I = OuterUnwindDest->begin();
+  for (unsigned i = 0, e = UnwindDestPHIValues.size(); i != e; ++i, ++I) {
+    PHINode *outerPhi = cast<PHINode>(I);
+    PHINode *innerPhi = PHINode::Create(outerPhi->getType(), phiCapacity,
+                                        outerPhi->getName() + ".lpad-body",
+                                        insertPoint);
+    innerPhi->addIncoming(outerPhi, OuterUnwindDest);
+  }
+
+  // Create a phi for the exception value...
+  InnerExceptionPHI = PHINode::Create(exn->getType(), phiCapacity,
+                                      "exn.lpad-body", insertPoint);
+  replaceAllInsnUsesWith(exn, InnerExceptionPHI);
+  selector->setArgOperand(0, exn); // restore this use
+  InnerExceptionPHI->addIncoming(exn, OuterUnwindDest);
+
+  // ...and the selector.
+  InnerSelectorPHI = PHINode::Create(selector->getType(), phiCapacity,
+                                     "selector.lpad-body", insertPoint);
+  replaceAllInsnUsesWith(selector, InnerSelectorPHI);
+  InnerSelectorPHI->addIncoming(selector, OuterUnwindDest);
+
+  // All done.
+  return InnerUnwindDest;
+}
+
+/// [LIBUNWIND] Try to forward the given call, which logically occurs
+/// at the end of the given block, as a branch to the inner unwind
+/// block.  Returns true if the call was forwarded.
+bool InvokeInliningInfo::forwardEHResume(CallInst *call, BasicBlock *src) {
+  Function *fn = dyn_cast<Function>(call->getCalledValue());
+  if (!fn || fn->getName() != "llvm.eh.resume")
+    return false;
+
+  // If this fails, maybe it should be a fatal error.
+  BasicBlock *dest = getInnerUnwindDest();
+  if (!dest) return false;
+
+  // Make a branch.
+  BranchInst::Create(dest, src);
+
+  // Update the phis in the destination.  They were inserted in an
+  // order which makes this work.
+  addIncomingPHIValuesForInto(src, dest);
+  InnerExceptionPHI->addIncoming(call->getArgOperand(0), src);
+  InnerSelectorPHI->addIncoming(call->getArgOperand(1), src);
+
+  return true;
 }
 
 /// [LIBUNWIND] Check whether this selector is "only cleanups":
@@ -134,7 +260,7 @@ static bool HandleCallsInBlockInlinedThroughInvoke(BasicBlock *BB,
 
     // LIBUNWIND: merge selector instructions.
     if (EHSelectorInst *Inner = dyn_cast<EHSelectorInst>(CI)) {
-      EHSelectorInst *Outer = findSelectorForLandingPad(Invoke.getUnwindDest());
+      EHSelectorInst *Outer = Invoke.getOuterSelector();
       if (!Outer) continue;
 
       bool innerIsOnlyCleanup = isCleanupOnlySelector(Inner);
@@ -172,48 +298,41 @@ static bool HandleCallsInBlockInlinedThroughInvoke(BasicBlock *BB,
     // First, split the basic block.
     BasicBlock *Split = BB->splitBasicBlock(CI, CI->getName()+".noexc");
 
-    bool skipNextBlock = false;
+    // Delete the unconditional branch inserted by splitBasicBlock
+    BB->getInstList().pop_back();
 
-    // LIBUNWIND: If this is a call to @_Unwind_Resume, just branch
+    // LIBUNWIND: If this is a call to @llvm.eh.resume, just branch
     // directly to the new landing pad.
-    if (isUnwindResume(CI->getCalledValue())) {
+    if (Invoke.forwardEHResume(CI, BB)) {
-      BranchInst::Create(Invoke.getUnwindDest(), BB->getTerminator());
-
       // TODO: 'Split' is now unreachable; clean it up.
 
       // We want to leave the original call intact so that the call
       // graph and other structures won't get misled.  We also have to
       // avoid processing the next block, or we'll iterate here forever.
-      skipNextBlock = true;
+      return true;
-
-    // Otherwise, create the new invoke instruction.
-    } else {
-      ImmutableCallSite CS(CI);
-      SmallVector<Value*, 8> InvokeArgs(CS.arg_begin(), CS.arg_end());
-      InvokeInst *II =
-        InvokeInst::Create(CI->getCalledValue(), Split, Invoke.getUnwindDest(),
-                           InvokeArgs.begin(), InvokeArgs.end(),
-                           CI->getName(), BB->getTerminator());
-      II->setCallingConv(CI->getCallingConv());
-      II->setAttributes(CI->getAttributes());
-    
-      // Make sure that anything using the call now uses the invoke!  This also
-      // updates the CallGraph if present, because it uses a WeakVH.
-      CI->replaceAllUsesWith(II);
-
-      Split->getInstList().pop_front();  // Delete the original call
     }
 
-    // Delete the unconditional branch inserted by splitBasicBlock
+    // Otherwise, create the new invoke instruction.
-    BB->getInstList().pop_back();
+    ImmutableCallSite CS(CI);
+    SmallVector<Value*, 8> InvokeArgs(CS.arg_begin(), CS.arg_end());
+    InvokeInst *II =
+      InvokeInst::Create(CI->getCalledValue(), Split,
+                         Invoke.getOuterUnwindDest(),
+                         InvokeArgs.begin(), InvokeArgs.end(),
+                         CI->getName(), BB);
+    II->setCallingConv(CI->getCallingConv());
+    II->setAttributes(CI->getAttributes());
+    
+    // Make sure that anything using the call now uses the invoke!  This also
+    // updates the CallGraph if present, because it uses a WeakVH.
+    CI->replaceAllUsesWith(II);
+
+    Split->getInstList().pop_front();  // Delete the original call
 
     // Update any PHI nodes in the exceptional block to indicate that
     // there is now a new entry in them.
     Invoke.addIncomingPHIValuesFor(BB);
-    
+    return false;
-    // This basic block is now complete, the caller will continue scanning the
-    // next one.
-    return skipNextBlock;
   }
 
   return false;

test/Transforms/Inline/inline_invoke.ll

@@ -18,7 +18,7 @@ declare i32 @llvm.eh.selector(i8*, i8*, ...) nounwind
 
 declare i32 @llvm.eh.typeid.for(i8*) nounwind
 
-declare void @_Unwind_Resume(i8*)
+declare void @llvm.eh.resume(i8*, i32)
 
 declare i32 @__gxx_personality_v0(...)
 
@@ -51,7 +51,7 @@ lpad:
           to label %invoke.cont2 unwind label %terminate.lpad
 
 invoke.cont2:
-  call void @_Unwind_Resume(i8* %exn) noreturn
+  call void @llvm.eh.resume(i8* %exn, i32 %eh.selector) noreturn
   unreachable
 
 terminate.lpad:
@@ -82,22 +82,27 @@ catch:
   br label %ret
 
 eh.resume:
-  call void @_Unwind_Resume(i8* %exn) noreturn
+  call void @llvm.eh.resume(i8* %exn, i32 %eh.selector) noreturn
   unreachable
 }
 
-; CHECK: define void @test0_out()
+; CHECK:    define void @test0_out()
-; CHECK: [[A:%.*]] = alloca %struct.A,
+; CHECK:      [[A:%.*]] = alloca %struct.A,
-; CHECK: [[B:%.*]] = alloca %struct.A,
+; CHECK:      [[B:%.*]] = alloca %struct.A,
-; CHECK: invoke void @_ZN1AC1Ev(%struct.A* [[A]])
+; CHECK:      invoke void @_ZN1AC1Ev(%struct.A* [[A]])
-; CHECK: invoke void @_ZN1AC1Ev(%struct.A* [[B]])
+; CHECK:      invoke void @_ZN1AC1Ev(%struct.A* [[B]])
-; CHECK: invoke void @_ZN1AD1Ev(%struct.A* [[B]])
+; CHECK:      invoke void @_ZN1AD1Ev(%struct.A* [[B]])
-; CHECK: invoke void @_ZN1AD1Ev(%struct.A* [[A]])
+; CHECK:      invoke void @_ZN1AD1Ev(%struct.A* [[A]])
-; CHECK: call i32 (i8*, i8*, ...)* @llvm.eh.selector(i8* {{%.*}}, i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*), i32 0, i8* bitcast (i8** @_ZTIi to i8*))
+; CHECK:      call i32 (i8*, i8*, ...)* @llvm.eh.selector(i8* {{%.*}}, i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*), i32 0, i8* bitcast (i8** @_ZTIi to i8*))
 ; CHECK-NEXT: invoke void @_ZN1AD1Ev(%struct.A* [[A]])
 ; CHECK-NEXT:   to label %[[LBL:[^\s]+]] unwind
 ; CHECK: [[LBL]]:
 ; CHECK-NEXT: br label %[[LPAD:[^\s]+]]
-; CHECK: [[LPAD]]:
+; CHECK:      ret void
-; CHECK-NEXT: call i8* @llvm.eh.exception()
+; CHECK:      call i8* @llvm.eh.exception()
 ; CHECK-NEXT: call i32 (i8*, i8*, ...)* @llvm.eh.selector(i8* {{%.*}}, i8* bitcast (i32 (...)* @__gxx_personality_v0 to i8*), i8* bitcast (i8** @_ZTIi to i8*))
+; CHECK-NEXT: br label %[[LPAD]]
+; CHECK: [[LPAD]]:
+; CHECK-NEXT: phi i8* [
+; CHECK-NEXT: phi i32 [
+; CHECK-NEXT: call i32 @llvm.eh.typeid.for(