Phi-3-Small-8K-Instruct
Microsoft · 2024-05-21 · 7.4B parameters
Phi-3 Small 8K Instruct is Microsoft's open-weight, instruction-tuned 7B-class decoder-only language model, released on May 21, 2024 with an 8,192-token context. Its 32-layer architecture alternates dense and block-sparse grouped-query attention and uses LayerNorm, RoPE, and GEGELU; the checkpoint was post-trained with supervised fine-tuning and direct preference optimization. Microsoft reports 4.8 trillion training-data tokens and an October 2023 data cutoff, so the rounded token disclosure and month-only cutoff remain null in the structured fields.