aitorrent/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF-torrent

This model is a refined version of the Meta Llama 3 family of large language models, specifically the 8B parameter variant. It has been generated using a methodology that "inhibits" the model's ability to express refusal by orthogonalizing the strongest refusal directions. This means that while it is not guaranteed to never refuse a request, it is tuned to minimize refusal behavior while maintaining the original model's knowledge and training intact.

Key Features

Orthogonalization: The model's weights have been manipulated to reduce refusal behavior, making it more responsive to user requests.
Abliteration: A play-on-words combining "ablation" and "obliterated," referring to the removal of specific features through orthogonalization.
Methodology: This approach is more surgical and data-efficient compared to fine-tuning, allowing for precise control over model behavior.
GGUF Quants: The model is available in various quantization formats (fp16, q8_0, q6_0, q4, q3_k_m) to accommodate different hardware and performance requirements.

Advantages

Preserves Original Knowledge: The model retains the original training and knowledge while removing undesirable refusal behavior.
Data Efficiency: Ablation requires less data compared to fine-tuning, making it a more efficient approach.
Flexibility: This methodology can be used in conjunction with fine-tuning for more comprehensive model refinement.

Quirkiness Awareness

As this methodology is new, the model may exhibit interesting quirks. Users are encouraged to experiment with the model and report any observed quirks to help understand the side effects of orthogonalization.

Community Engagement

The developer is open to feedback and encourages others to explore and improve upon this methodology. Support is available through the Cognitive Computations Discord and the Community tab.

Original Model Credit: https://huggingface.co/failspy/Meta-Llama-3-8B-Instruct-abliterated-v3-GGUF