Blog

Research, engineering, and insights from Kroonen AI.

2026-03-17

Distributed Checkpoint Failures on 2× RTX 4090 for Local LLM Pretraining

Technical note on how FSDP checkpoint saves reproducibly deadlock NCCL on consumer GPUs without NVLink. 26 WandB runs documenting the boundary between what works and what fails.

pretraining distributed-training nvidia rtx-4090

2026-03-15

Mapping the Mind of Qwen 3.5 9B: A Sparse Autoencoder for Mechanistic Interpretability

We release a sparse autoencoder trained on the internal activations of Qwen 3.5 9B, enabling feature-level inspection of what the model represents. Zero dead features, 16,384 interpretable dimensions, trained on a single RTX 4090.

interpretability sparse-autoencoder research