Hatchet

Hatchet is a distributed workflow orchestrator designed for building resilient and scalable data pipelines, particularly for AI agents and context engineering. It allows developers to define tasks and workflows as code using language-native SDKs (Python, TypeScript, Go), ensuring versionable, reusable, and testable atomic functions. The platform focuses on low-latency, high-throughput workloads, with features like smart assignment rules for rate limits, fairness, and priorities, and durable logging for every task invocation. Hatchet addresses common challenges in AI and data processing, such as keeping vector databases and knowledge graphs up-to-date, orchestrating complex AI agent behaviors, and parallelizing massive data processing tasks. It offers automatic retries, intelligent rate limiting, checkpoint recovery, and built-in eventing for human-in-the-loop signaling. The orchestration engine can be used as a managed service or self-hosted, with workers deployed on various container platforms, scaling automatically based on workload. It's ideal for scale-ups and enterprises needing robust, fault-tolerant, and high-performance workflow management.