Repository navigation
"First-Index" Bias #736
Description
In the assign_labels function under evaluation.py, we have the following code snippet:
# Compute proportions of spike activity per class.
proportions = rates / rates.sum(1, keepdim=True)
proportions[proportions != proportions] = 0 # Set NaNs to 0
# Neuron assignments are the labels they fire most for.
assignments = torch.max(proportions, 1)[1]
Does this introduce a first-index bias? That is, if proportions is a row of all zeros (which happens if a neuron hasn't fired for any class yet), torch.max doesn't throw an error, but identifies the maximum value as 0 and returns the index of its first occurrence, which is always index 0.
This potentially creates a synthetic over-representation of the first class. Specifically, we can get:
- Skewed Voting: inactive neurons contribute "votes" for Label 0 during inference, leading to an artificial boost in accuracy for the first class and decreased accuracy for others.
- Training Misinterpretation: Early in training, before neurons have specialized, the model will appear to have a strong preference for Label 0, misleading anyone interested about the learning progress.
Potential fix is maybe mask out the neurons with zero activity?
Activity
Hananel-Hazan commentedon Feb 11, 2026
Great catch, the torch.max behavior defaulting to index 0 for silent neurons definitely introduces a bias.
Masking out the inactive neurons, or assigning them a neutral label like -1, sounds like the correct fix here to prevent them from voting during inference.
Are you interested in opening a PR for this?
Machuka commentedon Feb 12, 2026
PR now open :)
Hananel-Hazan commentedon Feb 12, 2026
Thank you for making the PR! #737 great job.
Question, don't you think that now the bias moved from the 0 index to last assignment (-1 index)?
Machuka commentedon Feb 13, 2026
Yeah, it's not an elegant solution, but -1 can be filtered for. Additionally,
all_activityandproportion_weightingnaturally "filter" out the -1 when iterating throughn_labels. Alternatively, we could randomly assign labels. I guess it comes down to what's the most biologically plausible solutionHananel-Hazan commentedon Feb 16, 2026
Lets go with randomly assign labels.
Hananel-Hazan commentedon Jun 14, 2026
#763 close