Search Results for "data transformation"
Sort By:
Showing 53 open source projects for "data transformation"
View related business solutions-
Data management solutions for confident marketingVerify, deduplicate, manipulate, and assign records automatically to keep your CRM data accurate, complete, and ready for business.
-
Iris Powered By Generali - Iris puts your customer in control of their identity.Iris Identity Protection API sends identity monitoring and alerts data into your existing digital environment – an ideal solution for businesses that are looking to offer their customers identity protection services without having to build a new product or app from scratch.
-
1
Data-Juicer
Data processing for and with foundation models
Data-Juicer is an open-source data processing and augmentation framework designed to enhance the quality and diversity of datasets for machine learning tasks. It includes a modular pipeline for scalable data transformation. -
2
The Data Engineering Handbook
Links to everything you'd ever want to learn about data engineering
...It includes beginner and intermediate boot camps, interview guides, data cleaning and transformation resources, and curated lists of newsletters and industry communities, making it useful both for self-study and technical interview preparation. The repository is actively maintained and widely starred, reflecting its role as a go-to reference for newcomers and experienced practitioners alike. -
3
Pixeltable
Data Infrastructure providing an approach to multimodal AI workloads
Pixeltable is an open-source Python data infrastructure framework designed to support the development of multimodal AI applications. The system provides a declarative interface for managing the entire lifecycle of AI data pipelines, including storage, transformation, indexing, retrieval, and orchestration of datasets. Unlike traditional architectures that require multiple tools such as databases, vector stores, and workflow orchestrators, Pixeltable unifies these functions within a table-based abstraction. ... -
4
AI Data Science Team
An AI-powered data science team of agents
AI Data Science Team is a Python library and agent ecosystem designed to accelerate and automate common data science workflows by modeling them as specialized AI “agents” that can be orchestrated to perform tasks like data cleaning, transformation, analysis, visualization, and machine learning. It provides a modular agent framework where each agent focuses on a step in the typical data science pipeline — for example, loading data from CSV/Excel files, cleaning and wrangling messy datasets, engineering predictive features, building models with AutoML, connecting to SQL databases, and producing visual outputs — all driven by natural language or programmatic instructions. ... -
Run applications fast and securely in a fully managed environmentRun frontend and backend services, batch jobs, deploy websites and applications, and queue processing workloads without the need to manage infrastructure.
-
5
pdfly
CLI tool to extract (meta)data from PDF and manipulate PDF files
A Python library designed for manipulating PDF files with functionalities for extraction, transformation, and document generation. -
6
GraphRAG
A modular graph-based Retrieval-Augmented Generation (RAG) system
The GraphRAG project is a data pipeline and transformation suite that is designed to extract meaningful, structured data from unstructured text using the power of LLMs. -
7
Databend
Cloud-native open source data warehouse for analytics and AI queries
...Databend provides a unified engine capable of handling analytics, vector search, and full-text search within a single platform. Databend supports SQL-based workflows and enables real-time data ingestion, transformation, and analysis through streaming and task orchestration features. With its cloud-native design and distributed architecture, Databend can run both as a self-hosted system or within managed environments to power data analytics, AI workloads, and large-scale data. -
8
graphify
AI coding assistant skill (Claude Code, Codex, OpenCode, OpenClaw)
graphify is a data visualization and transformation tool designed to convert structured or semi-structured data into graph-based representations, enabling better understanding of relationships and dependencies. It focuses on building visual models such as nodes and edges that represent entities and their connections, making complex datasets easier to interpret. -
9
Apache Hamilton
Helps data scientists define testable self-documenting dataflows
...This approach encourages modular, testable, and maintainable data pipelines because each transformation is isolated and easily unit tested. The framework also automatically tracks lineage and metadata about how data is produced, which improves debugging, reproducibility, and transparency in data workflows. -
Get full visibility and control over your tasks and projects with Wrike.Wrike offers world-class features that empower cross-functional, distributed, or growing teams take their projects from the initial request stage all the way to tracking work progress and reporting results.
-
10
Instill Core
Instill Core is a full-stack AI infrastructure tool for data
Instill Core is an open-source, full-stack AI infrastructure platform designed to orchestrate data pipelines, machine learning models, and unstructured data processing into a unified, production-ready system. It provides an end-to-end solution that enables developers to build, deploy, and manage AI-powered applications without needing to manually stitch together multiple tools across the data and model lifecycle. The platform focuses heavily on handling unstructured data such as documents,... -
11
Pyper
Concurrent Python made simple
Pyper is a Python-native orchestration and scheduling framework designed for modern data workflows, machine learning pipelines, and any task that benefits from a lightweight DAG-based execution engine. Unlike heavier platforms like Airflow, Pyper aims to remain lean, modular, and developer-friendly, embracing Pythonic conventions and minimizing boilerplate. It focuses on local development ergonomics and seamless transition to production environments, making it ideal for small teams and... -
12
PyVista
3D plotting and mesh analysis through a streamlined interface
3D plotting and mesh analysis through a streamlined interface for the Visualization Toolkit (VTK). PyVista is a helper module for the Visualization Toolkit (VTK) that takes a different approach on interfacing with VTK through NumPy and direct array access. This package provides a Pythonic, well-documented interface exposing VTK’s powerful visualization backend to facilitate rapid prototyping, analysis, and visual integration of spatially referenced datasets. This module can be used for... -
13
Mito
AI-powered Jupyter spreadsheet that converts workflows into Python
Mito is an open source set of Jupyter extensions designed to speed up Python workflows and data analysis. It combines a spreadsheet-style interface with AI-assisted coding, allowing users to explore, clean, and transform data without switching tools. Mito includes a context-aware AI assistant that helps generate code, debug errors, and guide workflows directly inside Jupyter. Its spreadsheet layer supports familiar functions such as filters, pivot tables, and formulas, while automatically... -
14
Datumaro
Dataset Management Framework, a Python library and a CLI tool to build
Datumaro is a flexible Python-based dataset management framework and command-line tool for building, analyzing, transforming, and converting computer vision datasets in many popular formats. It supports importing and exporting annotations and images across a wide variety of standards like COCO, PASCAL VOC, YOLO, ImageNet, Cityscapes, and many more, enabling easy integration with different training pipelines and tools. Datumaro makes it easy to merge datasets, split them into... -
15
Fondant
Production-ready data processing made easy and shareable
Fondant is a modular, pipeline-based framework designed to simplify the preparation of large-scale datasets for training machine learning models, especially foundation models. It offers an end-to-end system for ingesting raw data, applying transformations, filtering, and formatting outputs—all while remaining scalable and traceable. Fondant is designed with reproducibility in mind and supports containerized steps using Docker, making it easy to share and reuse data processing components.... -
16
TorchAudio
Data manipulation and transformation for audio signal processing
The aim of torchaudio is to apply PyTorch to the audio domain. By supporting PyTorch, torchaudio follows the same philosophy of providing strong GPU acceleration, having a focus on trainable features through the autograd system, and having consistent style (tensor names and dimension names). Therefore, it is primarily a machine learning library and not a general signal processing library. The benefits of PyTorch can be seen in torchaudio through having all the computations be through PyTorch... -
17
TorchIO
Medical imaging toolkit for deep learning
TorchIO is an open-source Python library for efficient loading, preprocessing, augmentation and patch-based sampling of 3D medical images in deep learning, following the design of PyTorch. It includes multiple intensity and spatial transforms for data augmentation and preprocessing. These transforms include typical computer vision operations such as random affine transformations and also domain-specific ones such as simulation of intensity artifacts due to MRI magnetic field inhomogeneity... -
18
TorchRL
A modular, primitive-first, python-first PyTorch library
TorchRL is an open-source Reinforcement Learning (RL) library for PyTorch. TorchRL provides PyTorch and python-first, low and high-level abstractions for RL that are intended to be efficient, modular, documented, and properly tested. The code is aimed at supporting research in RL. Most of it is written in Python in a highly modular way, such that researchers can easily swap components, transform them, or write new ones with little effort. -
19
NeMo Retriever Library
Document content and metadata extraction microservice
NeMo Retriever Library is a scalable microservice framework designed for extracting, structuring, and enriching content from documents to support downstream generative AI applications. It processes various document types by splitting them into components such as text, tables, charts, and images, and then applies OCR and contextual analysis to convert them into structured data formats. The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient... -
20
E2M
E2M converts various file types (doc, docx, epub, html, htm, url
...By acting as a transformation layer, the software can support workflows such as converting data formats, integrating services, or bridging incompatible systems. The mirror hosted on SourceForge ensures that developers can continue accessing the project even if the primary repository becomes unavailable. -
21
Matrix
Multi-Agent daTa geneRation Infra and eXperimentation framework
Matrix is a distributed, large-scale engine for multi-agent synthetic data generation and experiments: it provides the infrastructure to run thousands of “agentic” workflows concurrently (e.g. multiple LLMs interacting, reasoning, generating content, data-processing pipelines) by leveraging distributed computing (like Ray + cluster management). The idea is to treat data generation as a “data-to-data” transformation: each input item defines a task, and the runtime orchestrates asynchronous, peer-to-peer agent workflows, avoiding global synchronization bottlenecks. ... -
22
Prompt Engineering Interactive Tutorial
Anthropic's Interactive Prompt Engineering Tutorial
Prompt-eng-interactive-tutorial is a comprehensive, hands-on tutorial that teaches the craft of prompt engineering with Claude through guided, executable lessons. It starts with the anatomy of a good prompt and moves into techniques that deliver the “80/20” gains—separating instructions from data, specifying schemas, and setting evaluation criteria. The course leans heavily on realistic failure modes (ambiguity, hallucination, brittle instructions) and shows how to iteratively debug prompts the way you would debug code. Lessons include building prompts from scratch for common tasks like extraction, classification, transformation, and step-by-step reasoning, with checkpoints that let you compare your outputs against solid baselines. ... -
23
Kubeflow pipelines
Machine Learning Pipelines for Kubeflow
...A pipeline component is a self-contained set of user code, packaged as a Docker image, that performs one step in the pipeline. For example, a component can be responsible for data preprocessing, data transformation, model training, and so on. -
24
Kornia
Open Source Differentiable Computer Vision Library
Kornia is a differentiable computer vision library for PyTorch. It consists of a set of routines and differentiable modules to solve generic computer vision problems. At its core, the package uses PyTorch as its main backend both for efficiency and to take advantage of the reverse-mode auto-differentiation to define and compute the gradient of complex functions. Inspired by existing packages, this library is composed by a subset of packages containing operators that can be inserted within... -
25
VoxelMorph
Unsupervised Learning for Image Registration
VoxelMorph is an open-source deep learning framework designed for medical image registration, a process that aligns multiple medical scans into a common spatial coordinate system. Traditional image registration techniques typically rely on optimization procedures that must be executed separately for each pair of images, which can be computationally expensive and slow. VoxelMorph approaches the problem using neural networks that learn to predict deformation fields that transform one image so...
