Skip to content

CryptoLabInc/envector-deployment

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

28 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

enVector - Encrypted Vector Search

License PyPI Docker Kubernetes Python OS

enVector is a product that provides secure vector search functionality using ES2 (Encrypted Similarity Search), which is based on Fully Homomorphic Encryption (FHE). This repository contains self-hosted deployment scripts and client SDK examples.

πŸš€ Features

  • πŸ” End-to-End Encryption: Secure vector search with FHE (Fully Homomorphic Encryption)
  • ⚑ High Performance: Optimized vector similarity search algorithms
  • πŸ—οΈ Microservices Architecture: Scalable and maintainable service design
  • 🐳 Multi-Platform Deployment: Docker Compose and Kubernetes (Helm) support
  • πŸ“± Python SDK: Easy-to-use client library for integration

πŸ”’ Security Features

  • Fully Homomorphic Encryption: Secure computation on encrypted data
  • Client-side Key Management: Secret key never leave the client
  • Encrypted Vector Storage: All vector data is encrypted at rest (at-rest)
  • Secure Search: Search operations performed on encrypted data (in-use)

πŸ“Š Performance

  • Vector Dimensions: Support for 32-4096 dimensional vectors
  • Search Speed: Optimized encrtyped similarity search algorithms
  • Scalability: Horizontal scaling with Kubernetes
  • GPU Support: Optional GPU acceleration for encrypted vector search

πŸ› οΈ Development

Prerequisites

  • Python 3.9-3.13
  • Docker and Docker Compose
  • Kubernetes cluster (for K8s deployment)
  • Helm 3.0+
  • Linux or macOS 11.0+

πŸ—οΈ Architecture

enVector consists of four main microservices:

  • es2e (Endpoint): Main API gateway and client interface
  • es2b (Backend): Service orchestration and metadata management
  • es2o (Orchestrator): Manages and schedules compute requests
  • es2c (Compute): Executes vector search and similarity computations

Infrastructure Dependencies

  • PostgreSQL: Metadata storage and management
  • MinIO: Vector data storage (S3-compatible)

πŸ“ Project Structure

envector-deployment/
β”œβ”€β”€ docker-compose/                  # Docker Compose deployment
β”‚   β”œβ”€β”€ docker-compose.envector.yml  # Core application services
β”‚   β”œβ”€β”€ docker-compose.infra.yml     # Postgres + MinIO (adds readiness deps to core)
β”‚   β”œβ”€β”€ docker-compose.gpu.yml       # GPU override for es2c
β”‚   β”œβ”€β”€ .env.example                 # environment variables for es2
β”‚   β”œβ”€β”€ start_envector.sh            # easy-to-use helper script
β”‚   └── README.md                    # Docker setup guide
β”œβ”€β”€ kubernetes-manifests/            # Kubernetes deployment
β”‚   β”œβ”€β”€ helm/                        # Helm chart for K8s
β”‚   β”‚   β”œβ”€β”€ Chart.yaml               # Chart metadata
β”‚   β”‚   β”œβ”€β”€ values.yaml              # Configurable values
β”‚   β”‚   └── templates/               # K8s manifest templates
β”‚   └── README.md                    # K8s deployment guide
└── notebooks/                       # Python SDK examples

πŸš€ Quick Start

⚠️ Important: Docker images are stored in private repositories. Please contact heaan for access credentials before proceeding with deployment.

Option 1: Docker Compose

Recommended for Development. See more details in docker-compose README.

Method A: Clone Repository

# Clone the repository
git clone https://github.com/CryptoLabInc/envector-deployment.git
cd envector-deployment/docker-compose

# Copy environment file (optional)
# If .env is missing, ./start_envector.sh will be created from .env.example automatically
cp .env.example .env

# Start services (performs preflight: Docker, PAT login if needed, license token)
./start_envector.sh
# OR docker compose -f docker-compose.envector.yml -f docker-compose.infra.yml -p envector up -d

Method B: Direct HTTP Usage

# Download and run directly from GitHub
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/docker-compose.envector.yml
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/docker-compose.infra.yml
curl -O https://raw.githubusercontent.com/cryptolabinc/envector-deployment/main/docker-compose/.env.example

# Copy environment file
cp .env.example .env

# Start services
docker compose -f docker-compose.envector.yml -f docker-compose.infra.yml -p envector up -d

Option 2: Kubernetes

Recommended for production.

Clone Repository

# Clone the repository
git clone https://github.com/CryptoLabInc/envector-deployment.git
cd envector-deployment

# Install Helm chart
helm install envector ./kubernetes-manifests/helm

# Check deployment status
kubectl get pods

# Access services
kubectl get svc

πŸ”§ Configuration

Environment Variables

Variable Description Default
ES2E_TAG es2e service image tag latest
ES2B_TAG es2b service image tag latest
ES2O_TAG es2o service image tag latest
ES2C_TAG es2c service image tag latest
ES2_LOG_LEVEL Logging level INFO
ES2E_HOST_PORT es2e external port 50050

Helm Values

Edit kubernetes-manifests/helm/values.yaml to customize:

  • Service ports and types
  • Resource limits and replicas
  • External database connections
  • Image repositories and tags

πŸ“š Python SDK Usage

  • Python Version: 3.9-3.13
  • OS: Linux/macOS 11.0+
pip install es2

Basic Setup

import es2

# Initialize connection
es2.init(
    host="localhost",
    port=50050,
    key_path="./keys",
    key_id="my_key"
)

# Create index
index = es2.create_index("my_index", dim=512)

# Insert vectors
vectors = [
    [0.001 * i for i in range(512)],
    [0.001 * i + 0.001 for i in range(512)],
]
index.insert(vectors, metadata=["doc1", "doc2"])

# Search
results = index.search(vectors[0], top_k=5)

Key Management

from es2.crypto import KeyGenerator, Cipher

# Generate FHE keys
keygen = KeyGenerator("./keys/my_key")
keygen.generate_keys()

# Create cipher for encryption/decryption
cipher = Cipher(dim=512, enc_key_path="./keys/my_key/EncKey.json")

πŸ“„ License

This project is proprietary software. For licensing information, please contact heaan.

🀝 Contributing

This is a proprietary software project. For contribution inquiries, please contact heaan.

πŸ“ž Support

πŸ”— Related Links


About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •