Home / FederAI / FederAI Specification
FederAI
From Every Device, A Smarter World.
FederAI is an open-source, ultra-efficient federated AI framework designed to make advanced artificial intelligence smaller, faster, more private, and accessible across everyday hardware. It combines adaptive low-bit neural architectures, sparse computation, federated learning, distributed inference, model compression, hardware optimization, privacy technologies, and a modular extension system.
FederAI uses a modular architecture. Core modules provide the essential capabilities for efficient AI execution, while optional plugin modules provide specialized functionality without unnecessarily increasing the size or complexity of the core system.
Vision
FederAI aims to create an AI ecosystem where intelligence does not depend entirely on centralized infrastructure.
The project is designed to enable AI models to run locally on laptops, desktops, mobile devices, edge hardware, embedded systems, browsers, and distributed networks. Devices can operate independently, collaborate with other nodes, or contribute computing resources to federated AI systems.
FederAI prioritizes:
- Efficient AI computation
- Low memory requirements
- Small model footprints
- Distributed intelligence
- Federated learning
- Privacy-preserving computation
- Hardware flexibility
- Open-source development
- Reproducible inference
- Energy-aware execution
- Extensible architecture
Core Modules
Adaptive Precision Module
The Adaptive Precision Module manages variable numerical precision throughout the model.
Capabilities include:
- Dynamic bitwidth selection
- Layer-level precision management
- Tensor-level precision management
- Precision-aware model optimization
- Runtime precision switching
- Mixed-precision inference
- Experimental sub-bit activation support
- Precision preservation for critical computations
The system can use extremely low precision where possible while retaining higher precision where additional numerical capacity is beneficial.
Sparse Intelligence Module
The Sparse Intelligence Module reduces unnecessary computation by identifying parameters, neurons, attention paths, and operations that do not need to be active for a particular workload.
Capabilities include:
- Dynamic sparsity
- Structured sparsity
- Unstructured sparsity
- Neuron pruning
- Parameter pruning
- Sparse tensor storage
- Zero-value computation skipping
- Token-dependent computation
- Sparse routing
- Runtime sparsity optimization
UltraBit Model Module
The UltraBit Model Module provides FederAI’s compact model representation.
The UltraBit Model format is designed for efficient storage, loading, transport, and execution of highly compressed models.
Capabilities include:
- Bit-packed weights
- Compact tensor representation
- Sparse indexing
- Hardware-aware tensor layouts
- Built-in compression
- Streaming model loading
- Memory-mapped execution
- Model metadata
- Precision metadata
- Hardware compatibility information
Model Optimization Module
The Model Optimization Module converts supported models into more efficient representations.
Capabilities include:
- Architecture analysis
- Adaptive quantization
- Weight compression
- Parameter pruning
- Weight deduplication
- Tensor packing
- Layer optimization
- Kernel selection
- Model restructuring
- Memory optimization
- Storage optimization
Model Intelligence Module
The Model Intelligence Module provides architectural techniques designed to improve capability while reducing unnecessary computation.
Capabilities include:
- Hybrid graph and token processing
- Micro-experts
- Lightweight mixture-of-experts routing
- Cross-layer attention reuse
- Compressed context representations
- Dynamic layer selection
- Adaptive computation depth
- Context-aware execution
- Learned sparse routing
Attention Optimization Module
The Attention Optimization Module reduces the computational and memory requirements associated with attention.
Capabilities include:
- Smart attention windows
- Adaptive attention spans
- Attention sparsification
- Cross-layer attention reuse
- Attention state compression
- Adaptive KV cache compression
- Context summarization
- Important-token prioritization
- Long-context optimization
Tokenization Module
The Tokenization Module manages efficient token representation and vocabulary optimization.
Capabilities include:
- Adaptive tokenization
- Vocabulary compression
- Token merging
- Rare-token optimization
- Compact token representations
- Domain-aware tokenization
- Tokenization benchmarking
Memory Module
The Memory Module manages model and runtime memory with the goal of minimizing RAM requirements.
Capabilities include:
- Memory mapping
- Streaming model execution
- Layer streaming
- Memory pooling
- Tensor reuse
- KV cache management
- Adaptive cache compression
- Memory-aware scheduling
- Automatic memory reclamation
Runtime Module
The Runtime Module provides the execution environment for FederAI models.
Capabilities include:
- Local inference
- Streaming inference
- Deterministic inference
- Dynamic computation
- Hardware-aware execution
- Runtime model loading
- Model switching
- Resource-aware scheduling
- CPU execution
- GPU execution
- NPU execution
Hardware Acceleration Module
The Hardware Acceleration Module provides optimized execution paths for different hardware architectures.
Supported targets can include:
- x86
- ARM
- RISC-V
- CPU
- GPU
- NPU
- WebAssembly
- WebGPU
Optimization interfaces can support:
- SIMD
- AVX
- AVX2
- AVX-512
- ARM NEON
- RISC-V Vector
- CUDA
- ROCm
- Vulkan
- Metal
- WebGPU
Kernel Optimization Module
The Kernel Optimization Module reduces execution overhead by generating and selecting efficient computational kernels.
Capabilities include:
- Kernel fusion
- Hardware-specific kernels
- Operation fusion
- SIMD optimization
- Memory access optimization
- Sparse kernels
- Low-bit kernels
- Dynamic kernel selection
- Runtime kernel benchmarking
Federated Learning Module
The Federated Learning Module enables distributed model training without requiring participating devices to transfer raw training data.
Capabilities include:
- Federated model training
- Distributed model updates
- Encrypted update exchange
- Privacy-preserving learning
- Local training
- Secure aggregation
- Client selection
- Node coordination
- Fault tolerance
- Model update compression
Distributed Compute Module
The Distributed Compute Module allows multiple devices to cooperate on AI workloads.
Capabilities include:
- Peer-to-peer inference
- Distributed inference
- Partial parameter sharding
- Model partitioning
- Compute offloading
- Opportunistic inference
- Node discovery
- Workload scheduling
- Network-aware routing
- Distributed model execution
Swarm Coordination Module
The Swarm Coordination Module allows FederAI nodes to coordinate available resources.
Nodes can share operational information such as:
- Available compute
- Memory capacity
- Network bandwidth
- Latency
- Hardware capabilities
- Current workload
- Energy availability
- Model availability
The scheduler can use this information to select appropriate nodes and distribute workloads efficiently.
Privacy Module
The Privacy Module provides privacy-preserving capabilities for local and distributed AI.
Capabilities include:
- Local differential privacy
- Privacy-preserving model updates
- No-trace inference
- Local-only execution
- Sensitive data isolation
- Configurable data retention
- Memory cleanup
- Privacy-aware scheduling
Security Module
The Security Module protects model execution, distributed communication, and federated updates.
Capabilities include:
- Secure communication
- Encrypted model updates
- Secure aggregation
- Integrity verification
- Authentication
- Authorization
- Model integrity checks
- Secure execution support
- Hardware enclave compatibility
Potential secure execution environments include:
- Intel SGX
- AMD SEV
- ARM TrustZone
Verifiable Compute Module
The Verifiable Compute Module provides optional mechanisms for verifying distributed computation.
Capabilities include:
- Computation verification
- Model integrity verification
- Execution attestations
- Optional zero-knowledge proofs
- Distributed result verification
- Node trust evaluation
Energy Awareness Module
The Energy Awareness Module measures and reports the resource requirements of AI workloads.
Capabilities include:
- Power monitoring
- Energy per inference
- Energy per token
- Tokens per watt
- Runtime efficiency measurements
- Hardware efficiency comparisons
- Optional carbon impact estimation
Deterministic Inference Module
The Deterministic Inference Module provides reproducible execution when deterministic behavior is required.
Capabilities include:
- Seed control
- Reproducible inference
- Deterministic kernels where supported
- Configuration recording
- Model version tracking
- Runtime configuration tracking
- Reproducibility reports
Evaluation Module
The Evaluation Module provides tools for measuring model quality and runtime behavior.
Capabilities include:
- Accuracy evaluation
- Factuality evaluation
- Hallucination testing
- Reasoning evaluation
- Bias testing
- Reliability testing
- Regression testing
- Model comparison
- Runtime benchmarking
Benchmark Module
The Benchmark Module measures performance across hardware and model configurations.
Metrics can include:
- Tokens per second
- Latency
- Throughput
- Memory usage
- Model size
- Active parameter count
- Sparsity
- Energy consumption
- Tokens per watt
- Distributed throughput
Model Surgery Module
The Model Surgery Module provides tools for modifying model architectures and optimizing existing models.
Capabilities include:
- Layer removal
- Layer merging
- Attention head modification
- Width adjustment
- Activation replacement
- Expert insertion
- Expert removal
- Parameter restructuring
- Precision reassignment
- Sparse restructuring
Operations should identify when retraining or additional calibration is required.
Model Fusion Module
The Model Fusion Module enables compatible models or model components to be combined.
Capabilities include:
- Layer fusion
- Parameter fusion
- Expert fusion
- Architecture-aware merging
- Model component selection
- Compatibility analysis
- Post-fusion evaluation
Plugin Module
FederAI supports optional plugins so specialized functionality does not need to be included in the core runtime.
Plugin capabilities can include:
- Custom tokenizers
- Custom quantizers
- Custom attention mechanisms
- Custom model formats
- Custom routing algorithms
- Custom hardware kernels
- Encryption modules
- Privacy modules
- Evaluation modules
- Model adapters
- Hardware adapters
- Distributed networking
- Specialized inference engines
Plugins should use documented interfaces and should not require unnecessary modifications to the core system.
Optional Plugin Modules
Hardware Plugin
Provides support for additional hardware architectures, accelerators, and vendor-specific optimizations.
Quantization Plugin
Provides experimental or specialized quantization methods without requiring them to be part of the core quantization engine.
Attention Plugin
Provides alternative attention mechanisms and experimental attention optimization techniques.
Tokenizer Plugin
Provides specialized tokenization systems for languages, domains, or constrained environments.
Routing Plugin
Provides alternative sparse routing and distributed workload scheduling strategies.
Privacy Plugin
Provides additional privacy mechanisms that can be installed when required by a deployment.
Security Plugin
Provides specialized authentication, encryption, attestation, and secure execution capabilities.
Model Format Plugin
Provides compatibility with additional model formats while keeping the UltraBit format as a native FederAI format.
Hardware Kernel Plugin
Provides externally maintained optimized kernels for specific processors, accelerators, or operating environments.
Federated Network Plugin
Provides alternative networking and federated coordination mechanisms.
Evaluation Plugin
Provides specialized evaluation suites for particular domains, model classes, or deployment requirements.
Developer Tools Plugin
Provides optional development interfaces, visualization tools, model inspectors, profilers, and debugging utilities.
FederAI, From Every Device, A Smarter World.
Specification Branding License (SBL)
Standard
- Fully AGPL-3.0+ compliant system
- Copyleft enforced for network deployments
- Required attribution:
- Roxanne Ardary
- https://www.roxanneardary.com/
Optional
- Specification Branding License (SBL)
- Attribution-free commercial deployment
- Pricing based on scale, usage, and deployment scope
- https://roxanneardary.com/federai/
License & Notice Requirements
FederAI is released under the GNU Affero General Public License v3.0 or later (AGPL-3.0+).
By contributing to this project, you agree that your contributions will also be released under this license.
Please note the following:
- All contributions must comply with the AGPL-3.0+ terms.
- Under Section 7 of the license, all redistributions, forks, and derivative works must preserve attribution to:
Roxanne Ardary and roxanneardary.com. - FederAI specifications are free to use with attribution. A Specification Branding License can be negotiated upon request.
- The project’s notice.md file tracks attribution requirements and contributor acknowledgments. Any update that adds new contributors or modifies attribution should also update
notice.md. - When submitting a pull request, ensure that any new files maintain the attribution headers where applicable.
- Network-deployed versions of this software must also remain fully AGPL-3.0+ compliant, including exposure of source code modifications when applicable under the license.
For full legal details, please refer to the AGPL-3.0+ license and the project’s notice.md file.
Notice – FederAI
Attribution Requirement: Under Section 7 of the AGPL 3.0+ license, all redistributions, forks, and derivative works, including network-deployed versions of this project, must provide attribution to Roxanne Ardary and roxanneardary.com.
Contributors
This file tracks contributors and their specific contributions to the project.
- Roxanne Ardary, roxanneardary.com – March 11, 2026
Created the repository for FederAI. Established the project architecture, licensing, and initial feature direction for the open-source federated sparse LLM initiative. - [Add other contributors here] – [Date]
[Describe contribution in one sentence]
License – FederAI
This repository is licensed under the GNU Affero General Public License v3.0 or later (AGPL-3.0+).
Key Points
- You are free to use, modify, and distribute the code.
- All redistributions, forks, and derivative works or network-deployed versions must also be licensed under AGPL-3.0+ and provide attribution to Roxanne Ardary and roxanneardary.com as required under Section 7 of the license.
- The software is provided “as is,” without warranty of any kind.
For the full license text, see GNU AGPL-3.0 License.