bartowski/Codestral-RAG-19B-Pruned-GGUF-torrent

bartowski/Codestral-RAG-19B-Pruned-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
FilenameQuant typeFile SizeDescription
Codestral-RAG-19B-Pruned-Q8_0.ggufQ8_020.32GBExtremely high quality, generally unneeded but max available quant.
Codestral-RAG-19B-Pruned-Q6_K.ggufQ6_K15.69GBVery high quality, near perfect, recommended.
Codestral-RAG-19B-Pruned-Q5_K_M.ggufQ5_K_M13.51GBHigh quality, recommended.
Codestral-RAG-19B-Pruned-Q5_K_S.ggufQ5_K_S13.17GBHigh quality, recommended.
Codestral-RAG-19B-Pruned-Q4_K_M.ggufQ4_K_M11.47GBGood quality, uses about 4.83 bits per weight, recommended.
Codestral-RAG-19B-Pruned-Q4_K_S.ggufQ4_K_S10.89GBSlightly lower quality with more space savings, recommended.
Codestral-RAG-19B-Pruned-IQ4_XS.ggufIQ4_XS10.26GBDecent quality, smaller than Q4_K_S with similar performance, recommended.
Codestral-RAG-19B-Pruned-Q3_K_L.ggufQ3_K_L10.09GBLower quality but usable, good for low RAM availability.
Codestral-RAG-19B-Pruned-Q3_K_M.ggufQ3_K_M9.26GBEven lower quality.
Codestral-RAG-19B-Pruned-IQ3_M.ggufIQ3_M8.66GBMedium-low quality, new method with decent performance comparable to Q3_K_M.
Codestral-RAG-19B-Pruned-Q3_K_S.ggufQ3_K_S8.30GBLow quality, not recommended.
Codestral-RAG-19B-Pruned-IQ3_XS.ggufIQ3_XS7.90GBLower quality, new method with decent performance, slightly better than Q3_K_S.
Codestral-RAG-19B-Pruned-IQ3_XXS.ggufIQ3_XXS7.40GBLower quality, new method with decent performance, comparable to Q3 quants.
Codestral-RAG-19B-Pruned-Q2_K.ggufQ2_K7.12GBVery low quality but surprisingly usable.
Codestral-RAG-19B-Pruned-IQ2_M.ggufIQ2_M6.56GBVery low quality, uses SOTA techniques to also be surprisingly usable.
Codestral-RAG-19B-Pruned-IQ2_S.ggufIQ2_S6.06GBVery low quality, uses SOTA techniques to be usable.
Codestral-RAG-19B-Pruned-IQ2_XS.ggufIQ2_XS5.72GBVery low quality, uses SOTA techniques to be usable.