2003-01-20 18:05:27 +00:00
|
|
|
|
2003-01-20 06:11:03 +00:00
|
|
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
|
|
|
<html><head><title>LLVM Test Results for $DateString</title></head>
|
|
|
|
|
|
|
|
|
|
<body bgcolor=white>
|
|
|
|
|
<center><font size=+3 face=Verdana><b>LLVM Test Results for $DateString</b></font></center>
|
|
|
|
|
<hr height=1>
|
|
|
|
|
|
|
|
|
|
<table width=100%>
|
|
|
|
|
<tr><td valign=top align=center>
|
|
|
|
|
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77">
|
|
|
|
|
<font size=+1><b>Sections:</b></font><br>
|
2003-01-22 16:13:10 +00:00
|
|
|
</td></tr><tr><td bgcolor="#FFCC99" align=center>
|
2003-01-20 06:11:03 +00:00
|
|
|
<a href="#Overview">Overview</a><br>
|
|
|
|
|
<a href="#Changes">Changes</a><br>
|
|
|
|
|
<a href="#Trends">Trends</a><br>
|
|
|
|
|
<a href="#Programs">Programs</a><br>
|
|
|
|
|
<a href="#Regression">Regression</a><br>
|
|
|
|
|
</td></tr></table></td></tr></table>
|
|
|
|
|
|
|
|
|
|
<p>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="5" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+1><b>Previous:</b></font><br>
|
|
|
|
|
</td></tr><tr><td bgcolor="#FFCC99">
|
|
|
|
|
$PrevDaysList
|
|
|
|
|
</td></tr></table></td></tr></table>
|
|
|
|
|
<p>
|
|
|
|
|
|
|
|
|
|
<font size=+1><b>Back to:</b></font><br>
|
|
|
|
|
<a href="..">Test Results</a><br>
|
|
|
|
|
<a href="../..">LLVM Page</a><p>
|
|
|
|
|
|
|
|
|
|
</td><td valign=top>
|
|
|
|
|
|
|
|
|
|
<center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+2 face=Verdana><b><a name="Overview">Today's Test Results Overview</font></b>
|
|
|
|
|
</td></tr></table></td></tr></table></center><p>
|
|
|
|
|
|
2003-08-18 19:59:28 +00:00
|
|
|
<!-- Running LOC graph -->
|
|
|
|
|
<table align=right>
|
|
|
|
|
<tr><td>
|
|
|
|
|
<a href="running_loc_large.png"
|
2003-08-19 14:27:44 +00:00
|
|
|
><img border=0 width=480 height=360 src="running_loc.png"></a>
|
2003-08-18 19:59:28 +00:00
|
|
|
</td></tr>
|
|
|
|
|
<tr><td align=center>Lines Of Code over Time<br>
|
|
|
|
|
<font size=-1><a href="running_loc_large.png">Click for larger view</a></font>
|
|
|
|
|
</td></tr>
|
|
|
|
|
</table>
|
|
|
|
|
|
2003-01-20 06:11:03 +00:00
|
|
|
<h2>CVS Tree Overview:</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li><a href="$DATE-CVS-Log.txt">CVS Checkout Log</a>
|
|
|
|
|
<ul>
|
2003-10-06 15:30:00 +00:00
|
|
|
<b>$NumDirsInCVS</b> dirs, <b>$NumFilesInCVS</b> files, <b>$LOC</b>
|
|
|
|
|
lines of code, checked out in <b>$CVSCheckoutTime</b> seconds<br></ul>
|
2003-01-20 06:11:03 +00:00
|
|
|
<li><a href="$DATE-Build-Log.txt">Compilation Log</a>
|
|
|
|
|
<ul>
|
2003-01-22 16:13:10 +00:00
|
|
|
$BuildError
|
2003-08-18 06:05:21 +00:00
|
|
|
Time to configure CVS tree: <b>$ConfigTime</b> seconds
|
|
|
|
|
(<b>$ConfigWallTime</b> seconds wall time)<br>
|
2003-01-22 20:35:59 +00:00
|
|
|
Time to build CVS tree: <b>$BuildTime</b> seconds
|
|
|
|
|
(<b>$BuildWallTime</b> seconds wall time)<br>
|
2003-01-20 06:11:03 +00:00
|
|
|
Number of object files compiled: <b>$NumObjects</b><br>
|
|
|
|
|
Number of libraries linked: <b>$NumLibraries</b><br>
|
|
|
|
|
Number of executables linked:<b> $NumExecutables</b><br>
|
|
|
|
|
</ul>
|
|
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<h2>Warnings during the build:</h2>
|
2003-02-02 17:02:20 +00:00
|
|
|
<ul>$WarningsList
|
|
|
|
|
</ul><p>
|
2003-01-20 06:11:03 +00:00
|
|
|
|
|
|
|
|
<br><br><center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+2 face=Verdana><b><a name="Changes">Changes from Yesterday</font></b>
|
|
|
|
|
</td></tr></table></td></tr></table></center><p>
|
|
|
|
|
|
|
|
|
|
<h2>Changes to CVS:</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>Users who committed to CVS: <b>$UserCommitList</b>
|
|
|
|
|
<li>Users who updated from CVS: <b>$UserUpdateList</b>
|
|
|
|
|
<li>Added Files: $AddedFilesList
|
|
|
|
|
<li>Modified Files: $ModifiedFilesList
|
|
|
|
|
<li>Removed Files: $RemovedFilesList
|
|
|
|
|
</ul><p>
|
|
|
|
|
|
2003-01-20 18:05:27 +00:00
|
|
|
<h2>Changes to Warnings:</h2>
|
|
|
|
|
<ul>
|
|
|
|
|
<li>Warnings Added: $WarningsAdded
|
|
|
|
|
<li>Warnings Removed: $WarningsRemoved
|
|
|
|
|
</ul><p>
|
|
|
|
|
|
2003-01-20 06:11:03 +00:00
|
|
|
<h2>Changes in the test suite:</h2>
|
|
|
|
|
<ul>
|
2003-01-22 20:35:59 +00:00
|
|
|
<li>New Tests: $TestsAdded
|
|
|
|
|
<li>Removed Tests: $TestsRemoved
|
|
|
|
|
<li>Newly passing tests: $TestsFixed
|
|
|
|
|
<li>Newly failing tests: $TestsBroken
|
2003-01-20 06:11:03 +00:00
|
|
|
</ul>
|
|
|
|
|
|
|
|
|
|
<br><br><center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+2 face=Verdana><b><a name="Trends">Changes over Time</font></b>
|
|
|
|
|
</td></tr></table></td></tr></table></center><p>
|
|
|
|
|
|
2003-11-08 20:48:39 +00:00
|
|
|
|
|
|
|
|
Here are some charts showing how the LLVM optimizer and code generators are
|
|
|
|
|
changing over time. For now we use the Olden benchmark suite to measure this,
|
|
|
|
|
but eventually we will switch to using SPEC CPU2000. All programs are run with
|
|
|
|
|
"LARGE_PROBLEM_SIZE" enabled. Click on any of the charts to get a larger
|
|
|
|
|
version.<p>
|
|
|
|
|
|
|
|
|
|
<h2>Compilation Measurements:</h2>
|
|
|
|
|
|
|
|
|
|
<table border="0" align=center>
|
|
|
|
|
<tr>
|
|
|
|
|
<td width=50% align=center>
|
|
|
|
|
<a href="running_Olden_bytecode_large.png"><img width=480 height=360 border=0 src="running_Olden_bytecode.png"></a><br>
|
|
|
|
|
Size of LLVM bytecode files
|
|
|
|
|
</td>
|
|
|
|
|
<td width=50% align=center>
|
|
|
|
|
<a href="running_Olden_machcode_large.png"><img width=480 height=360 border=0 src="running_Olden_machcode.png"></a><br>
|
|
|
|
|
Size of native machine code for each program (generated by the JIT)
|
|
|
|
|
</td></tr>
|
|
|
|
|
<tr>
|
|
|
|
|
<td align=center>
|
|
|
|
|
<a href="running_Olden_opt_time_large.png"><img width=480 height=360 border=0 src="running_Olden_opt_time.png"></a><br>
|
|
|
|
|
Time to run the LLVM optimizer on each program
|
|
|
|
|
</td>
|
|
|
|
|
<td></td></tr>
|
|
|
|
|
</table>
|
|
|
|
|
|
|
|
|
|
<h2>Program Execution Measurements:</h2>
|
|
|
|
|
|
|
|
|
|
<table border="0" align=center>
|
|
|
|
|
<tr>
|
|
|
|
|
<td width=50% align=center>
|
|
|
|
|
<a href="running_Olden_cbe_time_large.png"><img width=480 height=360 border=0 src="running_Olden_cbe_time.png"></a><br>
|
|
|
|
|
Execution time for CBE generated executable
|
|
|
|
|
</td>
|
|
|
|
|
<td width=50% align=center>
|
|
|
|
|
<a href="running_Olden_llc_time_large.png"><img width=480 height=360 border=0 src="running_Olden_llc_time.png"></a><br>
|
|
|
|
|
Execution time for the LLC generated executable
|
|
|
|
|
</td></tr>
|
|
|
|
|
|
|
|
|
|
<tr>
|
|
|
|
|
<td align=center>
|
|
|
|
|
<a href="running_Olden_jit_time_large.png"><img width=480 height=360 border=0 src="running_Olden_jit_time.png"></a><br>
|
|
|
|
|
Execution time for program in the JIT
|
|
|
|
|
</td>
|
|
|
|
|
<td></td></tr>
|
|
|
|
|
</table>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
2003-01-20 06:11:03 +00:00
|
|
|
|
|
|
|
|
<br><br><center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+2 face=Verdana><b><a name="Programs">Program Tests</font></b>
|
|
|
|
|
</td></tr></table></td></tr></table></center><p>
|
|
|
|
|
|
2003-01-22 16:13:10 +00:00
|
|
|
This section tests LLVM on a variety of programs in the test suite. This
|
2003-08-15 19:31:04 +00:00
|
|
|
includes benchmark suites like the Olden, McCat, Ptrdist, and SPEC benchmarks as
|
|
|
|
|
well as a few random programs with test inputs. This section is meant to track
|
|
|
|
|
how stable LLVM is as a whole. A failure in the execution of any test is marked
|
|
|
|
|
with an asterisk: `*'. The columns of the tables are:<p>
|
2003-01-22 16:13:10 +00:00
|
|
|
|
|
|
|
|
<ol>
|
2003-08-15 19:31:04 +00:00
|
|
|
<li><a name="Program">Program</a> - The name of the program for that row.</li>
|
|
|
|
|
<li><a name="GCCAS">GCCAS</a> - Time to run LLVM optimizers on the program.</li>
|
|
|
|
|
<li><a name="Bytecode">Bytecode</a> - The size of the bytecode for the
|
|
|
|
|
program</li>
|
|
|
|
|
<li><a name="Instrs">Instrs</a> - The number of LLVM instructions in the
|
|
|
|
|
compiled bytecode</li>
|
|
|
|
|
<li><a name="LLC<br>compile">LLC compile</a> - The time taken compile with
|
|
|
|
|
LLC (the static backend)</li>
|
|
|
|
|
<li><a name="JIT<br>codegen">JIT codegen</a> - The amount of time spent in the
|
|
|
|
|
JIT itself, instead of executing the program.</li>
|
|
|
|
|
<li><a name="Machine<br>code">Machine code</a> - The number of bytes of machine
|
|
|
|
|
code generated by the JIT.</li>
|
|
|
|
|
<li><a name="GCC">GCC</a> - The time taken to execute the program when compiled
|
2003-08-18 04:32:34 +00:00
|
|
|
with GCC -O2.</li>
|
2003-08-15 19:31:04 +00:00
|
|
|
<li><a name="CBE">CBE</a> - The time taken to execute the program after
|
2003-08-18 04:39:41 +00:00
|
|
|
compilation through the C backend, compiled with -O2.</li>
|
2003-08-15 19:31:04 +00:00
|
|
|
<li><a name="LLC">LLC</a> - How long does the program generated by the static
|
|
|
|
|
backend LLC take to execute </li>
|
|
|
|
|
<li><a name="JIT">JIT</a> - The amount of time spent running the
|
|
|
|
|
program with the JIT; this includes the code generation phase (listed above)
|
|
|
|
|
and actually running the program.</li>
|
2003-10-08 05:09:52 +00:00
|
|
|
<li><a name="GCC/LLC">GCC/LLC</a> - The speed-up of the LLC output vs the native
|
|
|
|
|
GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
|
|
|
|
|
<li><a name="GCC/CBE">GCC/CBE</a> - The speed-up of the CBE output vs the native
|
|
|
|
|
GCC output: greater than 1 is a speedup, less than 1 is a slowdown.</li>
|
2003-12-19 03:34:46 +00:00
|
|
|
<li><a name="LLC-LS">LLC-LS</a> - How long does the program generated by the static
|
|
|
|
|
backend LLC take to execute the program, when compiled with the linear scan
|
|
|
|
|
register allocator. This is temporary, for tuning.</li>
|
2003-01-22 16:13:10 +00:00
|
|
|
</ol><p>
|
|
|
|
|
|
2003-08-19 15:54:16 +00:00
|
|
|
A complete log of testing
|
2003-08-21 15:55:26 +00:00
|
|
|
<a href="$DATE-SingleSource-ProgramTest.txt.gz">SingleSource</a>,
|
|
|
|
|
<a href="$DATE-MultiSource-ProgramTest.txt.gz">MultiSource</a>, and
|
|
|
|
|
<a href="$DATE-External-ProgramTest.txt.gz">External</a> programs are
|
2003-08-19 15:54:16 +00:00
|
|
|
available</a> for further analysis.
|
2003-01-22 16:13:10 +00:00
|
|
|
|
2003-08-21 15:55:26 +00:00
|
|
|
<h2>Programs/External</h2>
|
|
|
|
|
|
|
|
|
|
<center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
$ExternalProgramsTable
|
|
|
|
|
</td></tr></table></center>
|
|
|
|
|
|
2003-01-22 16:13:10 +00:00
|
|
|
<h2>Programs/MultiSource</h2>
|
|
|
|
|
|
|
|
|
|
<center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
2003-08-18 06:05:21 +00:00
|
|
|
$MultiSourceProgramsTable
|
2003-01-22 16:13:10 +00:00
|
|
|
</td></tr></table></center>
|
|
|
|
|
|
|
|
|
|
<h2>Programs/SingleSource</h2>
|
|
|
|
|
|
2003-08-18 06:05:21 +00:00
|
|
|
<center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
$SingleSourceProgramsTable
|
|
|
|
|
</td></tr></table></center>
|
|
|
|
|
|
2003-01-22 16:13:10 +00:00
|
|
|
|
2003-01-20 06:11:03 +00:00
|
|
|
|
|
|
|
|
<br><br><center>
|
|
|
|
|
<table border="0" cellspacing="0" cellpadding="2"><tr><td bgcolor="#000000">
|
|
|
|
|
<table border="0" cellpadding="10" cellspacing="0"><tr><td bgcolor="#DDAA77"
|
|
|
|
|
<font size=+2 face=Verdana><b><a name="Regression">Regression Tests</font></b>
|
|
|
|
|
</td></tr></table></td></tr></table></center><p>
|
|
|
|
|
|
|
|
|
|
Not yet implemented<p>
|
|
|
|
|
|
|
|
|
|
</td></tr>
|
|
|
|
|
|
|
|
|
|
</html>
|
