Search Results for "transformer"
Sort By:
Showing 277 open source projects for "transformer"
View related business solutions-
Network Management Software and Tools for Businesses and Organizations | Auvik NetworksReduce IT headaches and save time with a proven solution for automated network discovery, documentation, and performance monitoring. Choose Auvik because you'll see value in minutes, and stay with us to improve your IT for years to come.
-
Inventory and Order Management Software for Multichannel SellersWe are the most powerful inventory and order management platform for Amazon, Walmart, and multichannel product sellers. Centralize orders, product information, and fulfillment operations to run more efficiently, sell more products, and stay compliant with marketplace requirements so you can grow profitably.
-
1
Transformer Engine
A library for accelerating Transformer models on NVIDIA GPUs
Transformer Engine (TE) is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper GPUs, to provide better performance with lower memory utilization in both training and inference. TE provides a collection of highly optimized building blocks for popular Transformer architectures and an automatic mixed precision-like API that can be used seamlessly with your framework-specific code. -
2
Transformer Explainer
Learn How LLM Transformer Models Work with Interactive Visualization
Transformer Explainer is an interactive visualization tool created to help users understand how transformer-based language models operate internally. The platform runs a lightweight GPT-2 model directly in the user’s browser and allows users to experiment with text prompts while observing the model’s internal operations. Through visual diagrams and interactive interfaces, the tool reveals how tokens are processed through layers such as embeddings, attention mechanisms, and feed-forward networks. ... -
3
Transformer Debugger
Tool for exploring and debugging transformer model behaviors
Transformer Debugger (TDB) is a research tool developed by OpenAI’s Superalignment team to investigate and interpret the behaviors of small language models. It combines automated interpretability methods with sparse autoencoders, enabling researchers to analyze how specific neurons, attention heads, and latent features contribute to a model’s outputs. -
4
Vision Transformer Pytorch
Implementation of Vision Transformer, a simple way to achieve SOTA
...It’s widely used as an educational reference for people learning transformers in vision and as a lightweight baseline for research prototypes. The project encourages experimentation—swap optimizers, change augmentations, or plug the transformer backbone into downstream tasks. -
Data management solutions for confident marketingVerify, deduplicate, manipulate, and assign records automatically to keep your CRM data accurate, complete, and ready for business.
-
5
spacy-transformers
Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy
...These techniques can be used to import knowledge from raw text into your pipeline, so that your models are able to generalize better from your annotated examples. You can convert word vectors from popular tools like FastText and Gensim, or you can load in any pre trained transformer model if you install spacy-transformers. You can also do your own language model pretraining via the spacy pre train command. You can even share your transformer or another contextual embedding model across multiple components, which can make long pipelines several times more efficient. To use transfer learning, you’ll need at least a few annotated examples for what you’re trying to predict. -
6
Transformers.jl
Julia Implementation of Transformer models
Transformers.jl is a Julia library that implements Transformer models for natural language processing tasks. Inspired by architectures like BERT, GPT, and T5, the library offers a modular and flexible interface for building, training, and using transformer-based deep learning models. It supports training from scratch and fine-tuning pretrained models, and integrates with Flux.jl for automatic differentiation and optimization. -
7
CTranslate2
Fast inference engine for Transformer models
CTranslate2 is a C++ and Python library for efficient inference with Transformer models. The project implements a custom runtime that applies many performance optimization techniques such as weights quantization, layers fusion, batch reordering, etc., to accelerate and reduce the memory usage of Transformer models on CPU and GPU. The execution is significantly faster and requires less resources than general-purpose deep learning frameworks on supported models and tasks thanks to many advanced optimizations: layer fusion, padding removal, batch reordering, in-place operations, caching mechanism, etc. ... -
8
RF-DETR
RF-DETR is a real-time object detection and segmentation
RF-DETR is an open-source computer vision framework that implements a real-time object detection and instance segmentation model based on transformer architectures. Developed by Roboflow, the project builds upon modern vision transformer backbones such as DINOv2 to achieve strong accuracy while maintaining efficient inference speeds suitable for real-time applications. The model is designed to detect objects and segment them within images or video streams using a unified detection pipeline. ... -
9
Megatron
Ongoing research training transformer models at scale
Megatron is a large, powerful transformer developed by the Applied Deep Learning Research team at NVIDIA. This repository is for ongoing research on training large transformer language models at scale. We developed efficient, model-parallel (tensor, sequence, and pipeline), and multi-node pre-training of transformer based models such as GPT, BERT, and T5 using mixed precision. -
Iris Powered By Generali - Iris puts your customer in control of their identity.Iris Identity Protection API sends identity monitoring and alerts data into your existing digital environment – an ideal solution for businesses that are looking to offer their customers identity protection services without having to build a new product or app from scratch.
-
10
Curated Transformers
PyTorch library of curated Transformer models and their components
State-of-the-art transformers, brick by brick. Curated Transformers is a transformer library for PyTorch. It provides state-of-the-art models that are composed of a set of reusable components. Supports state-of-the-art transformer models, including LLMs such as Falcon, Llama, and Dolly v2. Implementing a feature or bugfix benefits all models. For example, all models support 4/8-bit inference through the bitsandbytes library and each model can use the PyTorch meta device to avoid unnecessary allocations and initialization. -
11
Pyreft
ReFT: Representation Finetuning for Language Models
PyreFT is a tool by Stanford NLP for fine-tuning transformer models with an emphasis on efficient, resource-conserving training and customizability for NLP tasks. -
12
parcel/css
A CSS parser, transformer, and minifier written in Rust
A CSS parser, transformer, and minifier written in Rust. Parsing and minifying large files are completed in milliseconds, often with significantly smaller output than other tools. Many other CSS parsers treat property values as an untyped series of tokens. This means that each transformer that wants to do something with these values must interpret them itself, leading to duplicate work and inconsistencies. -
13
SeedVR
Repo for SeedVR2 & SeedVR
...SeedVR’s transformer-based design allows it to handle variable frame resolutions and lengths, and its architecture is optimized to overcome traditional limitations of windowed attention in high-resolution contexts. -
14
Detoxify
Trained models & code to predict toxic comments
Detoxify is a deep learning-based tool for detecting and filtering toxic language in online conversations, leveraging Transformer models for high accuracy. -
15
Minecraft Development for IntelliJ
Plugin for IntelliJ IDEA that gives special support for Minecraft mods
Experience first class support for all of the major Java Minecraft development platforms, including Bukkit and derivatives such as Spigot and Paper, Sponge, Forge, Fabric, MCP, Mixins, LiteLoader, BungeeCord, and Waterfall. It also provides in-depth support for Access Transformer and NBT files, and more. Because of this, you can install the plugin through IntelliJ's internal plugin browser. Navigate to File -> Settings -> Plugins and click the Browse Repositories... button at the bottom of the window. In the search box, simply search for Minecraft. You can install it from there and restart IntelliJ to activate the plugin. -
16
ACE-Step 1.5
The most powerful local music generation model
ACE-Step 1.5 is an advanced open-source foundation model for AI-driven music generation that pushes beyond traditional limitations in speed, musical coherence, and controllability by innovating in architecture and training design. It integrates cutting-edge generative techniques—such as diffusion-based synthesis combined with compressed autoencoders and lightweight transformer elements—to produce high-quality full-length music tracks with rapid inference times, capable of generating a complete song in seconds on modern GPUs while remaining efficient enough to run on consumer-grade hardware with minimal memory requirements. Beyond straightforward text-to-music synthesis, ACE-Step 1.5 enables flexible creative workflows, including tasks like cover generation, editing existing tracks, transforming vocals to background accompaniment, and stylistic personalization using low-rank adaptation from just a few example songs. -
17
model2Vec
Fast State-of-the-Art Static Embeddings
...The resulting models can be used for a wide range of tasks, including semantic search, clustering, classification, and retrieval-augmented generation systems. One of its key advantages is its simplicity, as it requires minimal dependencies and can generate embeddings extremely quickly compared to traditional transformer-based approaches. -
18
MoBA
MoBA: Mixture of Block Attention for Long-Context LLMs
...The approach allows language models to scale to significantly longer input contexts without the quadratic computational cost normally associated with transformer attention mechanisms. -
19
Z-Image
Image generation model with single-stream diffusion transformer
Z-Image is an efficient, open-source image generation foundation model built to make high-quality image synthesis more accessible. With just 6 billion parameters — far fewer than many large-scale models — it uses a novel “single-stream diffusion Transformer” architecture to deliver photorealistic image generation, demonstrating that excellence does not always require extremely large model sizes. The project includes several variants: Z-Image-Turbo, a distilled version optimized for speed and low resource consumption; Z-Image-Base, the full-capacity foundation model; and Z-Image-Edit, fine-tuned for image editing tasks. ... -
20
xFormers
Hackable and optimized Transformers building blocks
xformers is a modular, performance-oriented library of transformer building blocks, designed to allow researchers and engineers to compose, experiment, and optimize transformer architectures more flexibly than monolithic frameworks. It abstracts components like attention layers, feedforward modules, normalization, and positional encoding, so you can mix and match or swap optimized kernels easily. -
21
FlashAttention
Fast and memory-efficient exact attention
FlashAttention is a high-performance deep learning optimization library that reimplements the attention mechanism used in transformer models to be significantly faster and more memory-efficient than standard implementations. It achieves this by using IO-aware algorithms that minimize memory reads and writes, reducing the quadratic memory overhead typically associated with attention operations. The project provides implementations of FlashAttention, FlashAttention-2, and newer iterations optimized for modern GPU architectures such as NVIDIA Hopper and AMD accelerators. ... -
22
TabPFN
Foundation Model for Tabular Data
TabPFN is an open-source machine learning system that introduces a foundation model designed specifically for tabular data analysis. The model is based on transformer architectures and implements a prior-data fitted network that can perform supervised learning tasks such as classification and regression with minimal configuration. Unlike many traditional machine learning workflows that require extensive hyperparameter tuning and training cycles, TabPFN is pre-trained to perform inference directly on tabular datasets. ... -
23
TRIBE v2
A multimodal model for brain response prediction
...It is designed for in-silico neuroscience, enabling researchers to model how the brain responds to complex real-world inputs. The system integrates state-of-the-art encoders—including LLaMA for text, V-JEPA for video, and Wav2Vec-BERT for audio—into a unified Transformer architecture. This combined representation is mapped onto the cortical surface to predict fMRI responses across thousands of brain regions. TRIBE v2 allows researchers to simulate and analyze brain activity without requiring direct human experiments. Overall, it provides a powerful tool for studying perception, cognition, and multimodal processing in the brain. -
24
Intel LLM Library for PyTorch
Accelerate local LLM inference and finetuning
Intel LLM Library for PyTorch is an open-source acceleration library developed to optimize large language model inference and fine-tuning on Intel hardware platforms. Built as an extension of the PyTorch ecosystem, the library enables developers to run modern transformer models efficiently on Intel CPUs, GPUs, and specialized AI accelerators. The framework provides hardware-aware optimizations and low-precision computation techniques that significantly improve the performance of large language models while reducing memory consumption. IPEX-LLM supports a wide range of popular models, including architectures such as LLaMA, Mistral, Qwen, and other transformer-based systems. ... -
25
HunyuanDiT
Diffusion Transformer with Fine-Grained Chinese Understanding
HunyuanDiT is a high-capability text-to-image diffusion transformer with bilingual (Chinese/English) understanding and multi-turn dialogue capability. It trains a diffusion model in latent space using a transformer backbone and integrates a Multimodal Large Language Model (MLLM) to refine captions and support conversational image generation. It supports adapters like ControlNet, IP-Adapter, LoRA, and can run under constrained VRAM via distillation versions.
