Logo

Getting Started

  • Installation
    • Optional Dependencies
    • Platform-specific Instructions
  • Dependencies
    • Core
    • Optional

LLMs

  • Seamless Chat with LLMs
    • Authentication
  • LLM Inference
    • API Request Format
    • StreamProcessor
    • Authentication
  • Data Labeling using LLMs
  • Chunking
    • Fixed-size chunking
    • Semantic Chunking
    • Recursive chunking

MLMs

  • Train Text Classifiers
    • Using the Classifier
  • Pretrain MLMs
    • Tokenizer and Tokenization
    • Initialize an MLM and Train
  • Further Pretrain MLMs
    • Model pretrained using Langformers
    • Existing Model from HuggingFace

Embeddings

  • Embed Sentences
    • Textual Similarity
  • Semantic Search
    • Loading an Existing Database
  • Rerank Sentences
  • Mimick a Pretrained Model

Library Reference

  • Tasks
    • tasks
  • Classifiers
    • HuggingFaceClassifier
    • LoadClassifier
  • Embedders
    • HuggingFaceEmbedder
  • Generators
    • OllamaGenerator
    • HuggingFaceGenerator
    • StreamProcessor
  • Labellers
    • HuggingFaceDataLabeller
    • OllamaDataLabeller
  • Mimickers
    • EmbeddingMimicker
  • MLMs
    • MLMTokenizerDatasetCreator
    • HuggingFaceMLMCreator
  • Rerankers
    • CrossEncoder
  • Searchers
    • FaissSearcher
    • ChromaDBSearcher
    • PineconeSearcher

Development

  • Changelog
    • v0.5.0 (2024-05-04)
    • v0.4.0 (2024-04-17)
    • v0.3.1 (2024-04-16)
    • v0.3.0 (2024-04-14)
    • v0.2.0 (2024-04-10)
    • v0.1.0 (2024-04-08)
  • License
  • Contributing
    • Setting Up the Project
    • Ways to Contribute
    • Pull Request Guidelines
    • Documentation
    • License
langformers
  • Search

© Copyright 2025. Built with ❤️ for the future of language AI.

Built with Sphinx using a theme provided by Read the Docs.