6502/llvm-6502
1
0
Fork 0
mirror of https://github.com/c64scene-ar/llvm-6502.git synced 2025-06-15 05:24:01 +00:00
Code Issues Projects Releases Wiki Activity

Fix a integer overflow in SimplifyCFG's look up table formation logic.

Browse Source 
If the width is very large it gets truncated from uint64_t to uint32_t when
passed to TD->fitsInLegalInteger. The truncated value can fit in a register.
This manifested in massive memory usage or crashes (PR13946).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@164784 91177308-0d34-0410-b5e6-96231b3b80d8
This commit is contained in:
Benjamin Kramer
2012-09-27 18:29:58 +00:00
parent 593c0b08e6
commit 465251a7ce
2 changed files with 42 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
lib/Transforms/Utils/SimplifyCFG.cpp
View File

@ -3414,6 +3414,10 @@ bool SwitchLookupTable::WouldFitInRegister(const TargetData *TD,
return false;
// FIXME: If the type is wider than it needs to be, e.g. i8 but all values
// are <= 15, we could try to narrow the type.
// Avoid overflow, fitsInLegalInteger uses unsigned int for the width.
if (TableSize >= UINT_MAX/IT->getBitWidth())
return false;
return TD->fitsInLegalInteger(TableSize * IT->getBitWidth());
}