Models turn images and captions into vectors: lists of numbers used for classification and search. This study asks what the geometry of those vectors tells us about the tasks they can still perform.
A representation can look unusual geometrically while retaining useful information. It can also look broadly distributed while search results lose their relevance.
Distribution, recoverable labels, neighbour identity, and caption coverage are different properties. Evaluate the property the application actually needs.
FIG. 01 / FOUR QUESTIONS, ONE REPRESENTATIONConceptual illustration
Distribution
How do vectors spread?
Geometry reads the representation without using labels.
Labels
What can a classifier recover?
A fitted classifier tests whether label information is accessible.
Identity
Which neighbours stay nearby?
Exact identities can change even when coarse labels agree.
Relevance
How much of the target is found?
Finding one positive is different from recovering the positive set.
These are schematic examples, not measured embedding projections. The experiments below evaluate each property separately.
01 / Motivation
Search needs useful neighbours.
Imagine looking through a large collection of road images for situations similar to a rainy night on a city street. An embedding-based system ranks images by the similarity of their vectors.
The problem
Checking every result requires labels or human judgement. Geometric measurements are cheaper: they describe vector spread, direction, and dimensional structure without labels. But a warning from geometry does not tell us which task has stopped working.
The research question
Can label-free measurements tell us whether an embedding still supports classification and useful search?
The study deliberately separates what the vectors look like from what a classifier or retrieval system can do with them. Its contribution is an instrument comparison with explicit limitations, rather than a new embedding model or a universal quality score.
02 / Approach
Change the representation. Keep the question fixed.
Two complementary designs test the measurements: train controlled image models, then alter cached outputs from one pretrained model. They share the diagnostic questions, rather than a directly comparable score scale.
Controlled training
Seven variants. One training loop.
A Joint-Embedding Predictive Architecture (JEPA) predicts hidden image-region representations from visible regions. The encoder turns images into vectors; the predictor performs the hidden-region prediction. [1]
Change stop-gradient, a slowly updated target encoder, distribution regularization, or a projector. All seven conditions use the same training loop. Five matched seeds per condition give 35 completed runs, measured at the final 4,000-step checkpoint.
Fixed pretrained encoder
One model. Controlled vector changes.
EmbeddingGemma 2 maps images and text into a common 768-dimensional space. The study pins one revision, extracts and caches its outputs, and evaluates those vectors without retraining the encoder. [6]
Test imposed contraction, offsets, direction restriction and noise; compare compression; perturb numerical storage and scoring; then evaluate original image–caption pairs. These post-hoc changes do not establish what naturally occurs during training.
Prevents the target side of a prediction from being updated through that prediction’s gradient.
EMA target
A separate target encoder updated by an exponential moving average, a slowly changing copy of the main encoder.
SIGReg
Sketched Isotropic Gaussian Regularization: a published loss that encourages the embedding distribution to resemble a standard Gaussian. [2]
Projector
A temporary neural-network layer between the encoder and SIGReg. Evaluation reads the encoder, not this layer.
Shared architecture
One optimizer step: the masked branch is encoded and predicted, the full branch supplies the target latents, and the prediction loss is formed from the two.
Architecture schematic. Arrows show computation and loss inputs; spacing does not encode compute cost. The highlighting traces one optimizer step of Trainer.train_step for the selected condition and encodes no measured quantity. Evaluation always reads the context encoder.
Intervention matrix · seven conditions
Condition
Stop-gradient
EMA target
SIGReg
Projector
EMA reference
●
●
No
No
Stop-gradient only
●
No
No
No
SIGReg + stop-gradient
●
No
●
No
SIGReg
No
No
●
No
Projector + SIGReg + stop-gradient
●
No
●
●
Projector + SIGReg
No
No
●
●
Contracted control
No
No
No
No
Architecture motion explains computation in one training step. It is not a visualization of measured training dynamics. SIGReg follows a validated loss implementation, but this masked-prediction system is not a reproduction of LeJEPA. [2]
Data / Evaluation
Two datasets. Different definitions of similarity.
Driving attributes test coarse visual context. Recorded image–caption pairs test whether the corresponding image or captions are recovered. Neither is a complete definition of semantic relevance.
Driving context
BDD100K
Berkeley DeepDrive contains 100,000 driving videos; this project uses annotated still images. Weather, scene type, and time of day provide three label-based checks on the embeddings. [3]
The pretrained analysis uses a BDD100K-derived driving-scene subset with 2,000 training images and 983 validation images. Training images fit the classifier and PCA basis; validation images measure classification and retrieval.
Recorded correspondence
COCO image–caption pairs
COCO contains everyday-scene photographs with human annotations. This study samples 1,000 original-release validation image groups, five captions each, to test image-to-text and text-to-image retrieval. [8]
For a caption query, its recorded parent image is the positive. For an image query, its five recorded captions are positives. Other semantically valid matches may exist without being annotated.
No labels
Geometry
Variance: centered spread. Cosine: directional agreement. RankMe: raw singular-spectrum count. Participation ratio: centered covariance-spectrum count. Their definitions and blind spots differ. [4]Mathematical explanation ↓
Labels used
Classification
Freeze the encoder and fit a simple classifier, a linear probe. Accuracy counts correct predictions. Balanced accuracy averages class recall over the frozen eligible classes, so common classes do not dominate it.
Labels used
Search
Rank candidates by cosine similarity. BDD P@10 counts neighbours sharing the query’s attribute. COCO Hit@10 asks whether any recorded positive is found; caption recall@10 counts the fraction of all five captions recovered.
Training probes use raw embeddings with validation-selected regularization. Pretrained BDD probes standardize features. The studies also use different samples and output normalization: their absolute scores and variance scales are not an encoder benchmark. Chance baselines and class support are reported beside the relevant results. Training comparison ↓
03 / Training evidence
Less spread. Labels still recoverable.
Compare an EMA reference, with a slowly updated target and stop-gradient, against the same training loop with no prevention mechanism. These final-checkpoint results are means over five seeds. [5]
Embedding spread
Five-seed means. Logarithmic variance axis.
Labels recovered by a classifier
Five-seed means. Percent; time-of-day labels.
Time-of-day agreement in search
Five-seed means. Percent; time-of-day labels.
The unprotected control still predicts time of day at 91.37% accuracy. Its vector spread is much smaller, and its time-of-day neighbour agreement is 67.52%, compared with 89.49% for the reference.
The probe fits a classifier to recover labels. Retrieval uses the existing neighbour order. Their different readings show why a geometric warning, label decodability, and search utility must be kept separate.
A broader spectrum can contain less useful signal.
Hold the pretrained encoder fixed and change its output vectors. The severe interventions below use the .99 setting; direction restriction keeps eight directions fitted on training vectors only.
ContractionShrink each vector.
Common offsetMove all vectors together.
Rank restrictionKeep selected directions.
NoiseAdd unstructured variation.
Read the same row across both plots: the left measures spectrum breadth, the right measures weather agreement among neighbours. Red identifies the noise intervention. The dashed marks show random-neighbour agreement, 39.87%.
Spectrum breadth / RankMe
Raw singular-spectrum count. Higher means a broader spectrum, not necessarily more useful information.
Weather matches / P@10
P@10: fraction of ten neighbours sharing the weather label. For example, 57.86% means about 5.8 weather matches per ten results. Dashed line: random-neighbour floor 39.87%.
Noise raises RankMe while weather retrieval approaches chance. Spectral breadth describes a distribution; it does not identify whether its neighbours share useful labels.
Shrinking alone gives a different response.
Unchanged embeddings → severe positive contraction. Same sample, standardized classifier, and cosine retrieval.
Variance
0.257033→2.57033e-05
Weather balanced accuracy
63.02%→63.02%
Weather retrieval P@10
57.86%→57.86%
Positive scaling reduces variance quadratically, while cosine ordering and normalized rank ratios remain invariant. Standardization also removes positive feature scaling from the probe. A common offset leaves centered variance unchanged; standardization removes that offset too. These are established mathematical responses, demonstrated here on the fixed sample.
Raw RankMe and centered participation ratio can disagree because centering changes the spectrum being measured. Neither supplies a universal threshold. Absolute variance readings from the unnormalized training study do not transfer to normalized pretrained outputs.
Compare two ways of reducing native 768-dimensional embeddings to 512, 256, or 128 dimensions. Measure classification and retrieval separately for weather, scene, and time of day.
FIG. / TWO WAYS TO REDUCE A VECTORMethod schematic
Learned prefixes / MRL
Matryoshka Representation Learning trains nested prefixes to be useful. Keep the leading dimensions, then renormalize the shortened vector to unit length. [7]
768
512
256
128
Fitted projection / PCA
Principal Component Analysis learns directions of greatest variance from the training vectors. Subtract the training mean, project training and validation vectors into the same reduced basis, then normalize.
Center with training mean↓Project into training-fitted directions↓Normalize the reduced vector
Prefix widths encode the actual dimensional budgets. The PCA diagram shows operations, not measured computation time. Centering and coordinate construction also differ between the methods.
Native 768MRL / solid linePCA / dashed line
Weather / Balanced accuracy
Native 63.02%. At 128 dimensions: MRL 54.91%, PCA 62.36%.
Scene / Balanced accuracy
Native 69.52%. At 128 dimensions: MRL 66.70%, PCA 68.37%.
Time of day / Balanced accuracy
Native 76.44%. At 128 dimensions: MRL 74.19%, PCA 78.17%.
Weather / P@10
Native 57.86%. At 128 dimensions: MRL 56.33%, PCA 56.19%.
Scene / P@10
Native 62.44%. At 128 dimensions: MRL 60.78%, PCA 62.25%.
Time of day / P@10
Native 82.73%. At 128 dimensions: MRL 80.66%, PCA 84.00%.
No method leads on every attribute and endpoint. MRL 512 and 256 retain much of the measured native utility; at 128 dimensions, the ordering depends on what is measured.
For weather at 128 dimensions, PCA has higher balanced accuracy, while MRL has slightly higher neighbour-label agreement. Read the two curves separately: a classifier can use a direction that does not preserve the same cosine neighbours.
This is a comparison of existing compression methods on a fixed sample. It does not isolate the cause of their differences or establish population robustness, a new compression algorithm, or operational search-speed gains.
Compression axes show change from native in percentage points, with the same −10 to +5 range for every panel. Zero is the native result. Lines guide the eye between the tested dimensions, rather than predicting untested settings. Fixed-sample effects; no training-seed error bars. Exact compression table ↓.
06 / Neighbour stability
Small score errors can change who makes the top ten.
Store cached vectors with fewer bits or change similarity arithmetic, then repeat exact search. FP16 uses 16-bit floating point; INT8 uses 8-bit integer quantization. Neither condition changes the encoder’s inference precision.
FIG. / THE TOP-TEN BOUNDARYConceptual illustration
Reference order
10 Candidate A included
11 Candidate B outside
Nearly tied scoresA small perturbation can swap their order.
Perturbed order
10 Candidate B included
11 Candidate A outside
The boundary margin is the reference score gap between ranks 10 and 11. This schematic has no measured score scale. A query can keep the same attribute agreement while returning a different image.
Read across each row: how many queries returned different neighbours, and how much their similarity scores changed. Compare storage changes against the native reference; compare offset + FP16 against the common-offset reference. Error uses a logarithmic scale.
Queries with changed neighbours
Out of 983 queries. Native and common-offset rows each use their own unperturbed reference. Zero changes are reported explicitly.
Largest cosine-score error
Largest absolute cosine-score difference from the relevant reference. Logarithmic axis; smaller error alone does not guarantee stable neighbours.
Error magnitude alone does not explain neighbour stability. Under the common-offset stress, FP16 changes 195 top-10 sets despite a smaller maximum score error than native FP16, which changes five.
Native FP16 changes all lie in the smallest reference-margin bin. Reconstruction-based sufficient conditions cover 576 native FP16-gallery queries with no observed exception, but cover no INT8 or stressed queries. These are empirical checks of sufficient conditions, not rigorous floating-point certification.
Of the 195 stressed identity changes, weather P@10 stays unchanged for 131 queries, scene for 119, and time of day for 149. Coarse attribute agreement can hide neighbour turnover; it does not establish unchanged object layout, instance meaning, or safety relevance.
Most comparisons reconstruct stored vectors and score in FP64; the arithmetic-only condition normalizes and accumulates in FP32. Approximate nearest-neighbour search and deployment throughput were not tested. All numerical conditions ↓ · Boundary-margin evidence ↗.
07 / Image–caption retrieval
Finding one caption is not recovering the whole description set.
Use original COCO image–caption pairs with the same pretrained encoder. Compare native 768-dimensional vectors with learned 256- and 128-dimensional prefixes. An image has five recorded captions, so retrieval can be measured in several ways.
FIG. / ONE IMAGE, FIVE CAPTIONSCOCO sample / illustrative ranking
Ba woman standing on skiis while posing for the camera
CA woman in a red jacket skiing down a slope
DA young woman is skiing down the mountain slope.
Ea person on skis makes her way through the snow
Suppose the top ten return captions A, C, and E from this image, plus seven captions from other images. This example ranking explains the metrics; it is not a measured retrieval result.
Ten returned captions
A
other
other
C
other
other
E
other
other
other
Hit@10: yesAt least one positive was found.
Hit@1: yesThe first result is a positive.
Caption recall@10: 3/5Three of five captions were recovered.
The photograph and captions are from the evaluation sample. The illustrative ranking can score well on any-positive retrieval while missing some of the recorded caption set. Other returned captions may be semantically valid; recorded relevance is incomplete.
Native 768MRL 256MRL 128
Find at least one caption
Image → text Hit@10. A query counts as a hit when any of its five recorded captions appears in the top ten.
Recover the first result
Image → text Hit@1. The highest-ranked result must be one of the five recorded captions.
Recover more of the five captions
Image → text caption recall@10. Fraction of the five recorded captions recovered in the top ten.
At 128 dimensions, Hit@10 remains high while ranking and caption coverage lose more. Image-to-text Hit@10 falls 2.50 percentage points, compared with 10.60 points for Hit@1 and 11.34 points for caption recall@10.
Hit@10 change after compression
Compressed minus native, in percentage points. Dots: paired change; lines: conditional 95% bootstrap intervals. Left of zero means lower Hit@10; right means higher. Zero is the native result.
Reading the uncertainty
Compare the same image groups.
The intervals use 5,000 paired bootstrap draws of 1,000 parent image groups, carrying their captions together. They are conditional on the fixed gallery, prompts, and encoder.
A line spanning zero includes changes in both directions; it does not demonstrate equivalence. No intervals were declared for caption recall or identity overlap, so those changes remain descriptive.
These are evaluation-sample intervals, not encoder-training or independent-encoder uncertainty.
Geometry does not establish image–caption alignment.
A control reassigns the native image–caption correspondence while leaving vectors and rankings fixed. Hit@10 becomes 1.22% for text-to-image and 0.20% for image-to-text: unchanged geometry cannot establish paired relevance.
Category-level and paired utility broadly decline under stronger compression in this study. There is no strong opposing ordering where category utility is preserved while pairing is destroyed. Different retrieval directions also use different caption-role prefixes, so their scores cannot isolate modality asymmetry.
The useful outcome is a clearer evaluation practice: treat each diagnostic as an instrument with a definition and a scope, then check the task property being claimed.
Read geometry as geometry.
Spread and spectral counts can reveal contraction or directional concentration. They do not, alone, identify what labels are recoverable or what makes a useful neighbour.
Separate utility endpoints.
Classification, coarse neighbour agreement, exact neighbour identity, first-rank recovery, and multiple-caption coverage can respond differently to the same change.
Keep the evidence scope attached.
Training-seed variation, fixed-sample BDD effects, and conditional COCO bootstrap intervals answer different uncertainty questions.
What the study does not establish
Dataset scope
The BDD-derived validation subset contains 983 images. It has no foggy-weather, gas-station, or tunnel examples, so the results do not cover those categories.
Encoder scope
The pretrained results use one encoder, EmbeddingGemma 2. A second-encoder replication with Jina-CLIP-v2 could not be completed because its precision checks failed. The findings therefore have not been replicated across encoders.
Claim scope
The study establishes no universal quality predictor, full LeJEPA reproduction, independent encoder replication, operational search-speed benefit, or deployment certification. Familiar compression behavior is reported with its limitations.
Google DeepMind (2026). EmbeddingGemma 2 model card. Pinned model revision used for the fixed-encoder experiments, including image and text embeddings and normalized learned prefixes.
Lin, T.-Y., et al. (2014). Microsoft COCO: Common Objects in Context. ECCV. Everyday-scene imagery; the paired study uses original COCO2017 validation images and caption annotations.
Appendix / Supporting detail
Explore the complete evidence.
All seven training variants, extension tables, both caption-retrieval directions, paired training uncertainty, and the mathematics behind the dimensionality measurements.
All seven training variants
Explore the measurements
The plots show all seven conditions in the same order as the table. Each row is a model variant, including SIGReg with and without a projector. Each symbol is an individual seed; the short vertical marker is the five-seed mean. Use the selector to compare the three driving attributes.
EMA referenceStop-gradient onlySIGReg variantsContracted control
BDD100K, five training seeds, final step 4,000. Variance has a log axis; the semantic plots use 0–100% axes. Dashed lines show the majority floor for raw accuracy and random-neighbour expectation for retrieval. Balanced accuracy excludes classes below the ten-sample support floor.
The interactive plots could not load. The measured table below remains available; download the endpoint data.
HighLowHighest and lowest displayed means in each column, including ties.
All seven conditions · five-seed means · time of day
Condition
Total variance
RankMe
Linear-probe accuracy
Retrieval P@10
●EMA reference
108
78.36
92.71%High
89.49%High
◆Stop-gradient only
39.6
7.45
91.67%
86.70%
SIGReg + stop-gradient
178High
103.96High
91.76%
87.06%
SIGReg
3.24
24.45
91.11%Low
86.79%
Projector + SIGReg + stop-gradient
113
48.02
91.94%
87.53%
Projector + SIGReg
30.6
9.74
91.40%
85.83%
■Contracted control
6.55e-05Low
1.06Low
91.37%
67.52%Low
What the table shows
The highest geometry readings do not coincide with the highest task scores. SIGReg + stop-gradient has the largest total variance (178) and RankMe (103.96), while the EMA reference has the highest displayed time-of-day classification accuracy (92.71%) and retrieval P@10 (89.49%). Larger spread or a broader spectrum alone does not identify the most useful embeddings.
The control separates classification from retrieval. Its accuracy is 1.34 percentage points below the EMA reference, but its neighbour agreement is 21.97 points below. Recovering a label with a fitted classifier and finding neighbours that share that label test different properties.
A projector does not give a uniform retrieval increase. With SIGReg and stop-gradient, adding it raises the displayed mean P@10 by 0.47 percentage points; with SIGReg alone, it lowers the mean by 0.95 points. These are descriptive five-seed averages. See the paired uncertainty plots in the appendix for the pre-declared comparisons. [5]
Higher accuracy and P@10 mean more correct labels or greater neighbour agreement. Geometry columns describe distributions; their highest values do not automatically identify the most useful model. These are five-seed averages on one fixed evaluation split. The appendix shows paired differences and their 95% intervals for the five pre-declared comparisons.
Full result tables
Ochre High/Low marks the highest and lowest displayed values within each table column, including ties. These are numerical extrema, not significance or uniform model-quality verdicts.
Geometry and task responses
BDD driving-scene subset: Unchanged embeddings and severe interventions; 983 validation rows
Both caption-retrieval directions and neighbour overlap
Text → image / Hit@10
5,000 caption queries, 1,000 image candidates; one recorded parent per caption.
Text → image / Hit@1
Recorded parent recovered at the first rank.
Image → text / neighbour overlap
Percent of native top-10 identities retained. This is not relevance.
Text → image / neighbour overlap
Percent of native top-10 identities retained. This is not relevance.
COCO: 1,000 image groups and five captions per image; separate text roles by direction
Representation
T→I Hit@1 %
T→I Hit@10 %
I→T Hit@1 %
I→T Hit@10 %
I→T caption recall@10 %
Category P@10 %
Native 768d
73.86High
97.16High
84.30High
98.40High
80.24High
48.84High
MRL 256d
72.14
96.96
83.60
98.10
78.50
48.12
MRL 128d
62.50Low
93.82Low
73.70Low
95.90Low
68.90Low
44.88Low
Image-to-text uses document-role captions; text-to-image uses query-role captions. Directions differ in candidate count and number of recorded positives. Category P@10 is a separate image-only comparator over 75 eligible object categories. All category supports and retrieval results · Interpretation and controls.
How uncertain are these differences?
Compare the same seed, then average the differences.
Each row changes one mechanism. For each of the five seeds, subtract model B’s score from model A’s score: dᵢ = score(A, i) − score(B, i). This pairing removes variation shared by both runs, rather than treating them as unrelated experiments.
Read from the zero line. Right means a higher score for A; left means a lower score for A. The small dots are the five seed differences, the diamond is their mean, and the horizontal line is the 95% interval. A line spanning zero includes differences in both directions. A wider line means a less precise estimate under this protocol.
Classification accuracy
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
Neighbour agreement / P@10
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
Classification accuracy
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
Neighbour agreement / P@10
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
Classification accuracy
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
Neighbour agreement / P@10
Difference in percentage points. Dots: five paired seeds. Diamond: mean difference. Horizontal line: paired Student-t 95% interval. Zero: equal scores. Axis limits stay fixed across attributes for this metric.
What the interval covers
For the five differences, let d̄ be their mean and sd their sample standard deviation. The interval is d̄ ± 2.776 × sd / √5, using Student’s t distribution with four degrees of freedom. One percentage point means, for example, a change from 90% to 91%.
These intervals describe training-seed variation on one fixed evaluation split. They do not cover a different dataset, image split, or architecture. The five comparisons were declared before these runs; no correction for multiple comparisons is applied. The classifier-fitting settings were selected after training. EMA remains a reference, rather than a paired mechanism contrast, because it changes multiple factors. [5]
An embedding has 192 coordinates, but these can move together. If every image varies along one line, 192 coordinates describe only one independent direction. These measurements give a smooth count of how broadly the representation uses its directions, rather than counting every tiny nonzero direction equally.
Start with the matrix of image embeddings.
Stack n images as the rows of a matrix Z ∈ ℝn×d, with d = 192 coordinates per image. A spectrum lists the strength of independent, perpendicular directions in that matrix. Both measurements turn a spectrum into relative shares, then ask how evenly those shares are distributed.
01 / Participation ratio
Count directions of variation around the mean.
Subtract the mean embedding μ from each row to obtain X. The covariance matrix is C = XᵀX / (n − 1). Its eigenvalues λᵢ ≥ 0 are the variances along its independent directions. Their sum is total variance.
Why this makes sense: squaring the shares makes a dominant direction weigh heavily. If one direction holds everything, the sum of squared shares is 1 and PR is 1. If k directions each hold 1/k, that sum is 1/k and PR is k. Unequal shares give an intermediate count.
02 / RankMe
Count directions in the raw matrix.
Use Z without subtracting its mean. Singular value decomposition gives strengths σᵢ ≥ 0 for its perpendicular directions. Normalize these strengths into shares pᵢ, then calculate their entropy. [4]
Why exponentiate entropy? One direction has entropy 0, so its count is exp(0) = 1. With k equal shares, entropy is ln(k); exponentiating returns k. This puts the measurement in units of directions. Here ln is the natural logarithm and zero shares contribute zero to entropy.
A worked example: two directions, one much stronger.
Illustration only, not experiment data. Take a zero-mean matrix with singular values (3, 1, 0, 0). Because it is already centered, we can isolate the difference in weighting:
Covariance eigenvalues are proportional to squared singular values: (9, 1, 0, 0). Variance shares are (0.9, 0.1, 0, 0), giving PR = 1 / (0.9² + 0.1²) ≈ 1.22.
Both counts lie between one and two, because two directions are present but contribute unequally. They differ because variance squares the singular values, while RankMe uses their unsquared strengths.
What these counts cannot tell you
Centering matters. Adding the same offset vector to every embedding leaves centered covariance and PR unchanged. Raw RankMe can change because that shared mean contributes to the singular-value spectrum.
Magnitude matters separately. Multiplying every embedding by a small positive number leaves both normalized spectra unchanged, while total variance falls by the square of that number. A representation can therefore shrink without either count falling. Neither count identifies which directions encode useful labels; read them alongside variance, classification, and retrieval.
The equations show the ideal nonzero-spectrum case. The implementation adds 10−7 to RankMe’s normalized singular values for numerical stability and returns 1 for an exactly zero spectrum in either measurement. That convention is not evidence of useful information. [5]
