bartowski/Minitron-4B-Base-GGUF-torrent

FilenameQuant typeFile SizeSplitDescription
Minitron-4B-Base-f32.gguff3216.77GBfalseFull F32 weights.
Minitron-4B-Base-Q8_0.ggufQ8_04.46GBfalseExtremely high quality, generally unneeded but max available quant.
Minitron-4B-Base-Q6_K_L.ggufQ6_K_L3.83GBfalseUses Q8_0 for embed and output weights. Very high quality, near perfect, recommended.
Minitron-4B-Base-Q5_K_L.ggufQ5_K_L3.55GBfalseUses Q8_0 for embed and output weights. High quality, recommended.
Minitron-4B-Base-Q6_K.ggufQ6_K3.45GBfalseVery high quality, near perfect, recommended.
Minitron-4B-Base-Q4_K_L.ggufQ4_K_L3.28GBfalseUses Q8_0 for embed and output weights. Good quality, recommended.
Minitron-4B-Base-Q3_K_XL.ggufQ3_K_XL3.14GBfalseUses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability.
Minitron-4B-Base-Q5_K_M.ggufQ5_K_M3.06GBfalseHigh quality, recommended.
Minitron-4B-Base-Q5_K_S.ggufQ5_K_S2.99GBfalseHigh quality, recommended.
Minitron-4B-Base-Q4_K_M.ggufQ4_K_M2.70GBfalseGood quality, default size for must use cases, recommended.
Minitron-4B-Base-Q4_K_S.ggufQ4_K_S2.58GBfalseSlightly lower quality with more space savings, recommended.
Minitron-4B-Base-IQ4_XS.ggufIQ4_XS2.46GBfalseDecent quality, smaller than Q4_K_S with similar performance, recommended.
Minitron-4B-Base-Q3_K_L.ggufQ3_K_L2.45GBfalseLower quality but usable, good for low RAM availability.
Minitron-4B-Base-IQ3_M.ggufIQ3_M2.18GBfalseMedium-low quality, new method with decent performance comparable to Q3_K_M.
Minitron-4B-Base-IQ3_XS.ggufIQ3_XS2.06GBfalseLower quality, new method with decent performance, slightly better than Q3_K_S.
Minitron-4B-Base-Q2_K.ggufQ2_K1.90GBfalseVery low quality but surprisingly usable.