bartowski/Hathor_Respawn-L3-8B-v0.8-GGUF-torrent

bartowski/Hathor_Respawn-L3-8B-v0.8-GGUF · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
FilenameQuant typeFile SizeDescription
Hathor_Respawn-L3-8B-v0.8-Q8_0.ggufQ8_08.54GBExtremely high quality, generally unneeded but max available quant.
Hathor_Respawn-L3-8B-v0.8-Q6_K_L.ggufQ6_K_L6.85GBUses Q8_0 for embed and output weights. Very high quality, near perfect, recommended.
Hathor_Respawn-L3-8B-v0.8-Q6_K.ggufQ6_K6.59GBVery high quality, near perfect, recommended.
Hathor_Respawn-L3-8B-v0.8-Q5_K_L.ggufQ5_K_L6.05GBUses Q8_0 for embed and output weights. High quality, recommended.
Hathor_Respawn-L3-8B-v0.8-Q5_K_M.ggufQ5_K_M5.73GBHigh quality, recommended.
Hathor_Respawn-L3-8B-v0.8-Q5_K_S.ggufQ5_K_S5.59GBHigh quality, recommended.
Hathor_Respawn-L3-8B-v0.8-Q4_K_L.ggufQ4_K_L5.31GBUses Q8_0 for embed and output weights. Good quality, uses about 4.83 bits per weight, recommended.
Hathor_Respawn-L3-8B-v0.8-Q4_K_M.ggufQ4_K_M4.92GBGood quality, uses about 4.83 bits per weight, recommended.
Hathor_Respawn-L3-8B-v0.8-Q4_K_S.ggufQ4_K_S4.69GBSlightly lower quality with more space savings, recommended.
Hathor_Respawn-L3-8B-v0.8-IQ4_XS.ggufIQ4_XS4.44GBDecent quality, smaller than Q4_K_S with similar performance, recommended.
Hathor_Respawn-L3-8B-v0.8-Q3_K_XL.ggufQ3_K_XL4.78GBUses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability.
Hathor_Respawn-L3-8B-v0.8-Q3_K_L.ggufQ3_K_L4.32GBLower quality but usable, good for low RAM availability.
Hathor_Respawn-L3-8B-v0.8-Q3_K_M.ggufQ3_K_M4.01GBEven lower quality.
Hathor_Respawn-L3-8B-v0.8-IQ3_M.ggufIQ3_M3.78GBMedium-low quality, new method with decent performance comparable to Q3_K_M.
Hathor_Respawn-L3-8B-v0.8-Q3_K_S.ggufQ3_K_S3.66GBLow quality, not recommended.
Hathor_Respawn-L3-8B-v0.8-IQ3_XS.ggufIQ3_XS3.51GBLower quality, new method with decent performance, slightly better than Q3_K_S.
Hathor_Respawn-L3-8B-v0.8-IQ3_XXS.ggufIQ3_XXS3.27GBLower quality, new method with decent performance, comparable to Q3 quants.
Hathor_Respawn-L3-8B-v0.8-Q2_K_L.ggufQ2_K_L3.69GBUses Q8_0 for embed and output weights. Very low quality but surprisingly usable.
Hathor_Respawn-L3-8B-v0.8-Q2_K.ggufQ2_K3.17GBVery low quality but surprisingly usable.
Hathor_Respawn-L3-8B-v0.8-IQ2_M.ggufIQ2_M2.94GBVery low quality, uses SOTA techniques to also be surprisingly usable.
Hathor_Respawn-L3-8B-v0.8-IQ2_S.ggufIQ2_S2.75GBVery low quality, uses SOTA techniques to be usable.
Hathor_Respawn-L3-8B-v0.8-IQ2_XS.ggufIQ2_XS2.60GBVery low quality, uses SOTA techniques to be usable.