Modujo: Inspectable Model Research
Public research artifacts across continued pretraining, experimental instruction tuning, and model architecture exploration.
The research question
Explore sparse mixture-of-experts language models and publish checkpoints that others can inspect and use for further research.
Approach & published artifacts
The public repository separates continued-pretraining snapshots from experimental SFT artifacts. Its model cards document loading requirements, lineage, and limitations.
| Model | Total parameters | Active / token | Stage |
|---|---|---|---|
| Modujo-4B-A0.66B PT | ≈ 3.96B | ≈ 0.66B | Continued pretraining |
| Modujo-4B-A0.66B SFT | ≈ 3.96B | ≈ 0.66B | Experimental SFT |
| Modujo-1B-A0.75B | ≈ 1.00B | ≈ 0.75B | Base checkpoint |
The SFT checkpoint starts from an earlier PT snapshot; these are not equal-step comparisons. Active parameter count alone does not establish inference speed, memory usage, or quality.
Inspect the evidence
Training curves on the model site are labeled synthetic demonstrations. They are not measured training results. This page makes no claim of a completed benchmark advantage or customer outcome.