CLM LLM Benchmark Analysis and Statistical Model

About Dataset

LiveBench is a benchmark for large language models (LLMs) that prevents test contamination through monthly updates sourced from recent material. It ensures objective evaluation using verifiable answers and includes 18 tasks across six categories, with plans for continuous updates.

You can explore the website, the original paper, the code, and the datasets on livebench.ai. The data is made available by the authors through two main sources:

• LiveBench Website: Offers data and older versions with filtering options for easier exploration.

• Hugging Face: Provides detailed data for the latest version of the benchmark but does not include earlier evaluation versions.

While the website presents data in a visual and interactive format, it can be difficult to export the data for direct analysis. On the other hand, the data available on Hugging Face lacks information such as the model provider, earlier evaluations, and the most recent commits to the dataset available on the website.

This dataset aims to make the LiveBench time-series data available in the format presented on the website. To achieve this, it gathers and processes data from the website's GitHub repository and the files used by the live version of the website, ensuring that the data is always up-to-date.

import numpy as np 
import pandas as pd
import matplotlib.pyplot as plt

data = pd.read_csv('/Users/livebench.csv')

data.head()

data.size
87120

data.shape
(14520, 6)

data.columns
Index(['model', 'version_date', 'task', 'score', 'category', 'organization'], dtype='object')

data.isna()

data.isna().sum()

# CLT demonstration: sample means and histograms for different sample sizes
scores = data['score'].dropna().values
pop_mean = np.mean(scores)
pop_std = np.std(scores, ddof=0)

sample_sizes = [1, 5, 30, 100]
num_samples = 5000

fig, axes = plt.subplots(2, 2, figsize=(12, 8))
axes = axes.ravel()

for ax, n in zip(axes, sample_sizes):
    # draw sample means
    means = [np.mean(np.random.choice(scores, size=n, replace=True)) for _ in range(num_samples)]
    counts, bins, _ = ax.hist(means, bins=40, alpha=0.7, color='C0', edgecolor='black')
    bin_width = bins[1] - bins[0]
    # normal curve using CLT: mean = pop_mean, std = pop_std / sqrt(n)
    sigma = pop_std / np.sqrt(n)
    x = np.linspace(min(bins), max(bins), 200)
    normal_pdf = (1 / (sigma * np.sqrt(2 * np.pi))) * np.exp(-0.5 * ((x - pop_mean) / sigma) ** 2)
    ax.plot(x, normal_pdf * len(means) * bin_width, color='C3', lw=2, label='Normal approx')
    ax.axvline(pop_mean, color='k', linestyle='--', lw=1)
    ax.set_title(f"n={n}  sample mean μ={np.mean(means):.3f} σ={np.std(means, ddof=1):.3f}")
    ax.legend()

plt.suptitle("Central Limit Theorem: Distribution of Sample Means (score)", fontsize=14)
plt.tight_layout(rect=[0, 0.03, 1, 0.95])
plt.savefig('clt_histograms.png', dpi=150)
plt.show()

Conclusion

The visual demonstrates the Central Limit Theorem (CLT) in action by showing how the distribution of sample means evolves as the sample size increases. Across the four histograms (n = 1, 5, 30, 100), the results confirm the expected CLT behavior:

1. As sample size increases, the distribution becomes more normal

• At n = 1, the distribution is wide, irregular, and reflects the original population variability.

• By n = 30 and n = 100, the distribution of sample means becomes smooth, symmetric, and closely aligned with the normal curve.

2. Variability (σ) decreases sharply with larger samples

• σ drops from 26.003 → 11.723 → 4.732 → 2.664, showing that larger samples produce more stable and less noisy estimates.

• This confirms that sample means converge toward the true population mean as n grows.

3. The sample mean stays consistent across all sample sizes

• μ remains around 54.6–54.7, demonstrating that the estimator is unbiased and stable.

4. The normal approximation becomes extremely accurate at n ≥ 30

• The red curve (normal approximation) aligns almost perfectly with the histogram at n = 30 and n = 100, validating the CLT’s practical threshold.

The LLM benchmark clearly shows that LLM/CLM‑based statistical sampling behaves exactly as expected under the Central Limit Theorem:

• Small samples → high variance, non‑normal shape

• Large samples → low variance, strong normality, stable mean

This confirms that your sampling pipeline, randomization logic, and statistical modeling are functioning correctly and producing reliable, theoretically consistent results.