recorded test & check results
143,283 in tests_torch
of this run’s recorded results
One broad main run: 37786456684:1. Includes quality checks. This is not a count of distinct hand-written tests.
Example run total, not a guarantee for every PR. Written definitions, collected cases, and recorded executions are different counts.
Repository community.
stars
forks
subscribers
Actual notification watchers
open issues + PRs
Combined GitHub API count
GitHub watchers_count repeats stars; subscribers_count measures actual watchers.
Retrieved October 8. Metadata can change after this snapshot. Open-issues API count includes PRs.
Repository footprint.
tracked files
Python files
library source files
Under src/transformers/
documentation files
Under docs/
Git index at Transformers 4d536e1. Excludes Git internals and untracked work; counts overlap by category.
All file counts use git ls-files at the pinned snapshot. Python and directory subsets are not additive into total files.
Model implementation inventory.
| Measure | Count | Scope |
|---|---|---|
| Model source directories | 521 | Includes auto and shared support |
| Model test directories | 524 | Immediate model test directories |
| AutoConfig keys / class names | 725 / 722 | Submodels and aliases included |
| modeling_*.py files | 506 | Implementations include generated files |
| modular_*.py files | 300 | Sources for some generated implementations |
These are different repository measures. They do not count supported Hub checkpoints or independently tested architectures.
The generated registry has 719 mappings plus six explicit additions/aliases. Do not add modular and generated modeling file counts as independent models.
Static test tree inventory.
Python files
All parsed successfully
test-prefixed files
Some are shared bases or support
test_* definitions
Functions and methods
source lines
Includes comments and helpers
Before inheritance, parameterization, dependency gates, hardware gates, and runtime skips.
The AST scan covered every Python file in the local tests tree. It did not collect or execute every combination of dependencies and hardware. Static decorators do not establish a CPU/GPU percentage.
Test definitions by area.
under models/
in all other areas
| models | 10,414 |
| utils | 770 |
| shared / root | 463 |
| trainer | 439 |
| quantization | 435 |
| generation | 339 |
| pipelines | 263 |
| repo_utils | 205 |
| cli | 174 |
| other areas | 373 |
13,875 total definitions. Shared methods and parametrized cases expand runtime collection.
Other areas aggregate integrations, kernels, PEFT, tokenization, optimization, heterogeneity, exporters, TP, pipeline parallelism, conftest tests, and SageMaker. The full briefing lists all 20 areas.
Actions workflow records per day.
all workflow records
1,234 / day
PR CI records
98.4 / day
Includes reviews, issue/PR automation, and skipped workflows. Workflow records do not count each rerun attempt separately.
Daily API totals summed to 8,638. The weekly all-workflow request returned 2,500, so the reported total uses the daily responses. The PR CI weekly response matched the daily sum of 689.
Pull request throughput.
new PRs
October 1–7 UTC
new PRs / day
Seven complete calendar days
distinct PRs with CI
Rolling telemetry week
PRs with new runs / day
Rolling daily range: 17–60
New PRs by UTC date, October 1 → 7
A PR can have CI on multiple days. Created-PR counts and observed-CI participation use different populations and windows.
Creation counts use the GitHub issue search API with is:pr. Rolling telemetry starts October 1 at 14:50 UTC and ends October 8 at 14:50 UTC. Daily distinct PR participation: 40, 28, 17, 43, 60, 45, 56; do not add into weekly unique PRs.
Recorded PR-related results.
test-bearing run attempts
distinct numbered PRs
median results / run
Mixed run types; may be incomplete
Exact total: 68,096,064. P90 ≈ 198,054 / run. Repeated test execution counts again; quality checks and partial reports are included.
Filter observed run start times to the rolling week ending at the fixed cutoff. Deduplication preserves run, job, hardware, and PR. This is not unique logical tests, GPU-hours, or total test runtime.
Observed pytest executions.
deduplicated shard / trace executions
9,385 in the final 24-hour start cohort
| CPU | 7,287 |
| Single GPU | 107 |
| Multi-GPU | 108 |
Groups aggregate shards. These are observed executions and logical groups, not physical runners, GPU inventory, or fleet capacity.
All PR labels included. Shard executions deduplicate trace IDs and select by their associated workflow run start. An execution later than its workflow start belongs to that start cohort. Missing telemetry can undercount.
Measured runner allocations.
| ARC pool | CPU quota | Job RAM | Visible GPU count × VRAM |
|---|---|---|---|
| aws-m8i-l-cache | 3.8 vCPU | 14 GiB | CPU |
| aws-g5-4xlarge-cache | 22.8 vCPU | 83 GiB | 1 × A10G · ~22 GiB |
| aws-g5-12xlarge-cache | 45.6 vCPU | 176 GiB | 2 × A10G · ~22 GiB each |
Job allocations, not host specs. ARC fleet totals and configured limits are unavailable. The xGPU pool exposes two of the host’s four GPUs.
Fractional CPU values are cgroup quotas. Documented inventory entries include MI300/MI355 one-/two-GPU pools, a report pool, and GitHub-hosted Linux. AMD job memory/VRAM are unreported. Nine inventory types include a GitHub-hosted type, which is not ARC.
Observed average test durations.
Different workloads and environments. Observed averages are not guaranteed runtimes or a comparative hardware benchmark.
Fixed cutoff readings in seconds: Llama4 test_model_17b_16e_fp32 2487.732; Swin test_sdpa_can_compile_dynamic 2259.269; Cohere2MoE test_model_fp16 1959.160; CPU audio-classification example 230.368. Setup, downloads, compilation, and contention can contribute.
Observability data volume.
collector spans accepted / day
Spans, not unique test cases
trace payload received / day
About 9.3 GiB
Prometheus samples appended / day
Includes repeated scrapes of gauges
active Prometheus head series
Point-in-time head-series count
Counter increases are extrapolated estimates. Span, byte, sample, and series counts measure different entities.
Accepted spans: 16,638,329; sent spans: 16,636,290; trace bytes received: 9,978,214,886; samples appended: 775,746,919; active head series: 337,875. Filtering, buffering, and retries prevent naive exact reconciliation.
AI Slop gate
minimum account age
Enabled account check
maximum in 24 hours
Enabled activity check
checks must fail
Both enabled signals required
PRs carrying the label
Across all creation dates
Heuristics and label counts do not establish AI authorship, filter accuracy, or measured review-time savings.
The account check fails below 30 days; the fork check fails above seven forks within 24 hours. Trusted associations and exemption labels bypass checks. Automatic close/lock are disabled, not a measured count of closing events.
Agent (Serge) auto-fix
merged fixes
Across roughly 80 days
merged fixes / week
Approximate historical rate
verification repetitions
Before + after protocol
one investigation end to end
PVT v2 example
Historical figures and one illustrative investigation. The repetition protocol does not guarantee every candidate verified successfully.
Twenty-nine merged fixes over roughly 80 days are separate from the two-week funnel sample. The PVT v2 example had 36 turns, 56 seconds of LLM time, and approximately 28 minutes elapsed.