Experimental Results

← Back to Main Page ... Go to Next Page (Conclusion)

Results under Perfect Intervention

The comprehensive comparison results validate our measure-theoretic framework's ability to capture and exploit anti-causal structures in synthetic and real-world settings. The results highlight several key findings:

1. ACIA significantly outperforms all baselines

On CMNIST and RMNIST, ACIA achieves accuracy of 99.00%+, perfect environment independence (0.00), almost perfect interventional robustness (0.02 and 0.01) and low-level invariance (0.01 and 0.03), significantly surpassing other baselines. On Ball Agent, ACIA achieves 99.98% accuracy with almost perfect low-level invariance and interventional robustness. On the real-world Camelyon17, ACIA achieves the best test accuracy 84.40% while retaining the underlying causal properties.

2. Causal dynamic construction (Theorem 3) is confirmed

The low-level invariance in our results matches the theoretical expectation of environment-independent feature learning.

3. Interventional kernel invariance (Corollary) is empirically validated

The intervention robustness score validates the distinction between observational and interventional distributions.

4. Anti-Causal OOD generalization bound (Theorem) is substantiated

Test accuracy improvements over baselines across all datasets confirm our theoretical guarantees.

Results under Imperfect Intervention

Perfect/hard intervention completely disconnects intervened variables from their causes, while imperfect/soft intervention modifies causal mechanisms and maintains partial original dependencies. This experiment validates the effectiveness of ACIA against imperfect intervention on the studied datasets.

Dataset	CMNIST				RMNIST				Ball Agent				Camelyon17
Dataset	Acc	EI	LLI	IR	Acc	EI	LLI	IR	Acc	EI	LLI	IR	Acc	EI	LLI	IR
ACIA (Imperfect)	99.4	0.00	0.01	0.03	99.0	0.01	0.03	0.01	99.7	0.44	0.06	0.06	84.4	0.30	0.44	0.45

Imperfect intervention achieves similar results on the four datasets and metrics as perfect intervention. This supports our theoretical claim that ACIA can effectively handle both perfect and imperfect interventions via its interventional kernel formulation.

Visualizing Learnt Representations

Results on CMNIST

Figure: t-SNE visualization of ACIA representations on CMNIST. From left to right: (1) Low-level representations show initial digit clustering with color influence; (2) High-level representations show improved digit separation; (3) Environment visualization demonstrates removal of environment-specific information; (4) Parity analysis reveals clear separation between even and odd digits.

Figure demonstrates ACIA's ability to learn representations that perfectly capture the anti-causal structure and predict test data:

Panel 1: Low-level representations show clear digit-based clustering while retaining certain environment information (note that some colored images are mapped to digit clusters they don't belong to).
Panel 2: High-level representations improve digit cluster separation with clearer boundaries.
Panel 3: Environment visualization displays colored images from different environments mixed, confirming removal of environment-specific information.
Panel 4: Parity visualization reveals how ACIA organizes digits based on their mathematical properties—the alternating pattern between even digits (orange) and odd digits (blue) confirms ACIA preserves meaningful numerical relationships while eliminating spurious color correlations.

Results on RMNIST

Figure: t-SNE visualization of ACIA representations on RMNIST. From left to right: (1) Low-level representations show digit clustering but with rotation influence; (2) High-level representations with better digit boundaries; (3) Rotation angle visualization shows uniform distribution across the representation space; (4) Digit complexity reveals semantic organization by structural properties.

Similar patterns emerge in RMNIST:

Panel 1: Low-level representations show clear digit-based clustering but retain certain rotation information.
Panel 2: High-level representations with more distinct boundaries.
Panel 3: Rotation angle visualization displays uniform coloring across the entire representation space, confirming successful abstraction of rotation-specific information.
Panel 4: Digit complexity visualization reveals semantic organization where digits with similar structural properties cluster together:
- Simple (0,1,7): minimal stroke count (typically 1-2), more rotation-invariant features, lower topological complexity
- Medium (2,3,5): moderate stroke count (typically 2-3), mixed curves/lines, intermediate visual density
- Complex (4,6,8,9): most stroke count (3+), multiple curves/intersections, higher topological complexity

Results on Ball Agent

Figure: t-SNE visualization of ACIA representations on Ball Agent. From left to right: (1) Low-level representations display position-based organization with environmental mixing; (2) High-level representations show more pronounced position clustering; (3) Intervention visualization categorized by intervention patterns; (4) Prediction error shows areas of high accuracy (green) versus areas requiring improvement (red).

Ball Agent results demonstrate ACIA's handling of continuous labels and interventions:

Panel 1: Low-level representations display position-based organization with considerable mixing between position values.
Panel 2: High-level representations show more pronounced position-based clustering with clearer boundaries, demonstrating improved abstraction of spatial information.
Panel 3: Intervention visualization displays categorical distribution of intervention patterns (None, Single, Double, Multiple), revealing how different intervention types affect the latent space structure.
Panel 4: Prediction error visualization shows areas of high accuracy (green) versus areas requiring improvement (red), confirming that position-relevant information is preserved while achieving partial invariance to interventions.

Results on Camelyon17

Figure: t-SNE visualization of ACIA representations on Camelyon17. From left to right: (1) Low-level representations show partial tumor/normal tissue separation; (2) High-level representations with improved class boundaries; (3) Hospital visualization demonstrates mixing of environment-specific features; (4) Uncertainty analysis highlights regions of high confidence (green/yellow) versus regions requiring more evidence (red).

Real-world medical data confirms ACIA's practical applicability:

Panel 1: Low-level representations show separation between tumor and normal tissue samples, but with certain mixing.
Panel 2: High-level representations demonstrate more pronounced clustering with clearer boundaries between tissue types, particularly visible in the left-right separation.
Panel 3: Hospital visualization displays significant mixing between hospital sources despite their different staining protocols, confirming reduction of environment-specific information.
Panel 4: Uncertainty visualization highlights regions where the model maintains high confidence (green/yellow) versus areas requiring more evidence (red).