url/SkillOpt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Simplify main results webpage table

Browse Source
This commit is contained in:
Lliar-liar
2026-05-24 17:10:35 +00:00
parent 9a064f7c97
commit cad3ab2d19
2 changed files with 96 additions and 458 deletions
Download Patch File Download Diff File Expand all files Collapse all files
index.html
+48 -229
View File
@@ -306,8 +306,7 @@
}
.statement h3,
.panel h3,
.result-card h3 {
.panel h3 {
margin: 0 0 12px;
font-family: var(--display);
font-size: 1.45rem;
@@ -471,144 +470,6 @@
line-height: 1.55;
}
.results-grid {
display: grid;
grid-template-columns: repeat(3, minmax(0, 1fr));
gap: 12px;
}
.result-card {
min-height: 202px;
padding: 22px;
background: var(--panel);
border: 1px solid var(--line);
border-radius: 8px;
}
.result-card.featured {
color: #f8faf7;
background: var(--teal);
border-color: var(--teal);
}
.metric {
display: block;
margin: 10px 0 6px;
font-family: var(--display);
font-size: 3rem;
font-weight: 800;
line-height: 1;
}
.result-card p {
margin: 0;
color: var(--muted);
font-size: 0.96rem;
}
.result-card.featured p {
color: rgba(248, 250, 247, 0.84);
}
.model-gallery {
display: grid;
grid-template-columns: repeat(7, minmax(172px, 1fr));
gap: 10px;
margin-top: 16px;
overflow-x: auto;
padding-bottom: 4px;
}
.model-card {
min-width: 172px;
padding: 15px;
background: var(--panel);
border: 1px solid var(--line);
border-radius: 8px;
box-shadow: 0 10px 22px rgba(18, 24, 31, 0.08);
}
.model-card.highlight {
color: #f8faf7;
background: var(--black);
border-color: var(--black);
}
.model-card strong {
display: block;
min-height: 38px;
font-family: var(--mono);
font-size: 0.78rem;
line-height: 1.25;
}
.score-pair {
display: flex;
justify-content: space-between;
gap: 10px;
margin: 12px 0;
color: var(--quiet);
font-family: var(--mono);
font-size: 0.68rem;
text-transform: uppercase;
}
.score-pair b {
display: block;
margin-top: 3px;
color: var(--ink);
font-size: 0.95rem;
}
.model-card.highlight .score-pair {
color: rgba(248, 250, 247, 0.58);
}
.model-card.highlight .score-pair b {
color: #f8faf7;
}
.gain-meter {
height: 9px;
background: rgba(21, 25, 31, 0.12);
border-radius: 999px;
overflow: hidden;
}
.model-card.highlight .gain-meter {
background: rgba(248, 250, 247, 0.16);
}
.gain-meter i {
display: block;
height: 100%;
width: var(--w);
background: linear-gradient(90deg, var(--gold), var(--green));
border-radius: inherit;
}
.gain-label {
display: block;
margin-top: 8px;
color: var(--green);
font-family: var(--display);
font-size: 1.55rem;
font-weight: 800;
line-height: 1;
}
.model-card.highlight .gain-label {
color: #f4c542;
}
.result-note {
margin: 10px 0 0;
color: var(--muted);
font-family: var(--mono);
font-size: 0.72rem;
line-height: 1.55;
}
.table-wrap {
overflow-x: auto;
background: var(--panel);
@@ -620,7 +481,7 @@
table {
width: 100%;
border-collapse: collapse;
min-width: 820px;
min-width: 1040px;
font-family: var(--mono);
font-size: 0.78rem;
line-height: 1.35;
@@ -652,6 +513,10 @@
background: rgba(235, 238, 240, 0.62);
}
.harness-group td {
border-top: 2px solid var(--line-strong);
}
.num {
text-align: right;
white-space: nowrap;
@@ -1126,9 +991,7 @@
font-size: 4.1rem;
}
.results-grid,
.method-grid,
.model-gallery,
.transfer-grid,
.evolution-footnotes {
grid-template-columns: repeat(2, minmax(0, 1fr));
@@ -1166,9 +1029,7 @@
font-size: 2rem;
}
.results-grid,
.method-grid,
.model-gallery,
.transfer-grid,
.evolution-footnotes,
.steps {
@@ -1392,86 +1253,19 @@
<div>
<h2>SkillOpt improves GPT and Qwen students.</h2>
<p class="section-lede">
Direct-chat results are reported for seven target models, not only
GPT-5.5. The cross-model view below averages the six benchmark scores
in the main paper table, comparing no-skill execution with the final
SkillOpt skill for each student.
The table reports main-result gains across target models and
execution harnesses, comparing no-skill execution with the final
SkillOpt skill on held-out test splits.
</p>
</div>
</div>
<div class="results-grid">
<article class="result-card featured">
<h3>Covered students</h3>
<span class="metric">7</span>
<p>GPT-5.5, GPT-5.4 family, GPT-5.2, and two Qwen targets in direct chat.</p>
</article>
<article class="result-card">
<h3>Largest average lift</h3>
<span class="metric">+24.9</span>
<p>GPT-5.4-nano gains the most on average, showing strong benefit for weaker students.</p>
</article>
<article class="result-card">
<h3>Largest single lift</h3>
<span class="metric">+50.7</span>
<p>Qwen3.5-4B on ALFWorld, where the optimized skill turns procedural memory into a large gain.</p>
</article>
</div>
<div class="model-gallery" aria-label="Average direct-chat gains by student model">
<article class="model-card highlight">
<strong>GPT-5.5</strong>
<div class="score-pair"><span>No skill<b>58.8</b></span><span>SkillOpt<b>82.3</b></span></div>
<div class="gain-meter"><i style="--w: 94%;"></i></div>
<span class="gain-label">+23.5</span>
</article>
<article class="model-card">
<strong>GPT-5.4</strong>
<div class="score-pair"><span>No skill<b>59.7</b></span><span>SkillOpt<b>72.4</b></span></div>
<div class="gain-meter"><i style="--w: 51%;"></i></div>
<span class="gain-label">+12.8</span>
</article>
<article class="model-card">
<strong>GPT-5.4-mini</strong>
<div class="score-pair"><span>No skill<b>51.1</b></span><span>SkillOpt<b>63.8</b></span></div>
<div class="gain-meter"><i style="--w: 51%;"></i></div>
<span class="gain-label">+12.7</span>
</article>
<article class="model-card highlight">
<strong>GPT-5.4-nano</strong>
<div class="score-pair"><span>No skill<b>32.4</b></span><span>SkillOpt<b>57.4</b></span></div>
<div class="gain-meter"><i style="--w: 100%;"></i></div>
<span class="gain-label">+24.9</span>
</article>
<article class="model-card">
<strong>GPT-5.2</strong>
<div class="score-pair"><span>No skill<b>51.3</b></span><span>SkillOpt<b>67.9</b></span></div>
<div class="gain-meter"><i style="--w: 67%;"></i></div>
<span class="gain-label">+16.6</span>
</article>
<article class="model-card highlight">
<strong>Qwen3.5-4B</strong>
<div class="score-pair"><span>No skill<b>38.6</b></span><span>SkillOpt<b>57.9</b></span></div>
<div class="gain-meter"><i style="--w: 77%;"></i></div>
<span class="gain-label">+19.2</span>
</article>
<article class="model-card">
<strong>Qwen3.6-35B-A3B</strong>
<div class="score-pair"><span>No skill<b>55.9</b></span><span>SkillOpt<b>65.0</b></span></div>
<div class="gain-meter"><i style="--w: 37%;"></i></div>
<span class="gain-label">+9.1</span>
</article>
</div>
<p class="result-note">
Values are six-benchmark direct-chat averages computed from the main result matrix. Bars are scaled by average gain over no skill.
</p>
<div class="table-wrap" style="margin-top: 16px;">
<table aria-label="Direct chat gain heatmap by model and benchmark">
<div class="table-wrap">
<table aria-label="Main result gain heatmap by model, harness, and benchmark">
<thead>
<tr>
<th>Student model</th>
<th>Harness</th>
<th class="num">SearchQA</th>
<th class="num">Sheet</th>
<th class="num">Office</th>
@@ -1484,6 +1278,7 @@
<tbody>
<tr>
<td>GPT-5.5</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 19;">+9.6</td>
<td class="num heat" style="--heat: 77;">+38.9</td>
<td class="num heat" style="--heat: 77;">+39.0</td>
@@ -1494,6 +1289,7 @@
</tr>
<tr>
<td>GPT-5.4</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 12;">+6.2</td>
<td class="num heat" style="--heat: 42;">+21.1</td>
<td class="num heat" style="--heat: 25;">+12.8</td>
@@ -1504,6 +1300,7 @@
</tr>
<tr>
<td>GPT-5.4-mini</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 8;">+4.3</td>
<td class="num heat" style="--heat: 22;">+11.4</td>
<td class="num heat" style="--heat: 53;">+26.7</td>
@@ -1514,6 +1311,7 @@
</tr>
<tr>
<td>GPT-5.4-nano</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 37;">+19.0</td>
<td class="num heat" style="--heat: 16;">+8.2</td>
<td class="num heat" style="--heat: 66;">+33.7</td>
@@ -1524,6 +1322,7 @@
</tr>
<tr>
<td>GPT-5.2</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 22;">+11.2</td>
<td class="num heat" style="--heat: 37;">+18.9</td>
<td class="num heat" style="--heat: 42;">+21.5</td>
@@ -1534,6 +1333,7 @@
</tr>
<tr>
<td>Qwen3.5-4B</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 6;">+3.1</td>
<td class="num heat" style="--heat: 29;">+14.6</td>
<td class="num heat" style="--heat: 30;">+15.2</td>
@@ -1544,6 +1344,7 @@
</tr>
<tr>
<td>Qwen3.6-35B-A3B</td>
<td>Direct chat</td>
<td class="num heat" style="--heat: 15;">+7.6</td>
<td class="num heat" style="--heat: 18;">+9.3</td>
<td class="num heat" style="--heat: 2;">+1.2</td>
@@ -1552,6 +1353,28 @@
<td class="num heat" style="--heat: 44;">+22.4</td>
<td class="num heat-avg">+9.1</td>
</tr>
<tr class="harness-group">
<td>GPT-5.5</td>
<td>Codex</td>
<td class="num heat" style="--heat: 11;">+5.5</td>
<td class="num heat" style="--heat: 100;">+57.5</td>
<td class="num heat" style="--heat: 25;">+12.8</td>
<td class="num heat" style="--heat: 10;">+5.0</td>
<td class="num heat" style="--heat: 55;">+28.0</td>
<td class="num">N/A</td>
<td class="num heat-avg">+21.8</td>
</tr>
<tr>
<td>GPT-5.5</td>
<td>Claude Code</td>
<td class="num heat" style="--heat: 8;">+4.0</td>
<td class="num heat" style="--heat: 100;">+58.3</td>
<td class="num heat" style="--heat: 27;">+13.9</td>
<td class="num heat" style="--heat: 7;">+3.5</td>
<td class="num heat" style="--heat: 26;">+13.3</td>
<td class="num">N/A</td>
<td class="num heat-avg">+18.6</td>
</tr>
</tbody>
</table>
</div>
@@ -1561,6 +1384,7 @@
<thead>
<tr>
<th>Benchmark</th>
<th>Harness</th>
<th class="num">No skill</th>
<th class="num">Best non-SkillOpt baseline</th>
<th class="num">SkillOpt</th>
@@ -1571,6 +1395,7 @@
<tbody>
<tr>
<td>SearchQA</td>
<td>Direct chat</td>
<td class="num">77.7</td>
<td class="num">84.8</td>
<td class="num"><strong>87.3</strong></td>
@@ -1579,6 +1404,7 @@
</tr>
<tr>
<td>SpreadsheetBench</td>
<td>Direct chat</td>
<td class="num">41.8</td>
<td class="num">73.6</td>
<td class="num"><strong>80.7</strong></td>
@@ -1587,6 +1413,7 @@
</tr>
<tr>
<td>OfficeQA</td>
<td>Direct chat</td>
<td class="num">33.1</td>
<td class="num">66.9</td>
<td class="num"><strong>72.1</strong></td>
@@ -1595,6 +1422,7 @@
</tr>
<tr>
<td>DocVQA</td>
<td>Direct chat</td>
<td class="num">78.8</td>
<td class="num">90.6</td>
<td class="num"><strong>91.2</strong></td>
@@ -1603,6 +1431,7 @@
</tr>
<tr>
<td>LiveMathBench</td>
<td>Direct chat</td>
<td class="num">37.6</td>
<td class="num">52.0</td>
<td class="num"><strong>66.9</strong></td>
@@ -1611,6 +1440,7 @@
</tr>
<tr>
<td>ALFWorld</td>
<td>Direct chat</td>
<td class="num">83.6</td>
<td class="num">93.3</td>
<td class="num"><strong>95.5</strong></td>
@@ -1620,17 +1450,6 @@
</tbody>
</table>
</div>
<div class="split" style="margin-top: 16px;">
<article class="panel">
<h3>Codex harness</h3>
<p>With GPT-5.5 in a Codex-style execution harness, SkillOpt reaches 85.0 on SpreadsheetBench and 78.4 on LiveMathBench, outperforming no skill by +57.5 and +28.0 points respectively.</p>
</article>
<article class="panel">
<h3>Claude Code harness</h3>
<p>In the Claude Code-style harness, SkillOpt reaches 80.4 on SpreadsheetBench and 71.5 on OfficeQA, remaining stronger than EvoSkill in the reported harness block.</p>
</article>
</div>
</section>
<section class="section" id="ablations">