SpatialMind AI

An Intelligent Spatial Reasoning & 3D Gaussian Splatting Platform

---

SpatialMind AI is a high-performance WebGL platform designed to render, inspect, and converse with massive 3D Gaussian Splatting environments. It combines real-time spatial Level-of-Detail (LOD) tiling with in-memory multi-spectral transformations and a conversational 3D Spatial RAG search pipeline.

---

✨ Key Features

Feature	Description
60 FPS WebGL Engine	Renders 4.8M+ 3D Gaussian Splats at locked 60 FPS using an adaptive O(N log N) frustum plane tiling strategy
Multi-Spectral Vision	Server-side binary byte mutation engine alters raw 32-byte splat parameters in RAM for Satellite, LiDAR, Thermal IR, and X-Ray views — zero disk overhead
Conversational 3D RAG	Combines Qdrant Vector Search, Neo4j Scene Graphs, and LLMs (Llama 3 / GPT-4o) for natural language → 3D camera flights + GPU shader highlights
Render Quality Controls	Resolution scaling (720p → 4K) with real-time contrast, saturation, and sharpness post-processing
First-Person Navigation	WASD keyboard controls for immersive 3D environment exploration

---

🏗️ Architecture Overview

┌──────────────────────────────────────────────────────────────────┐
│                        CLIENT (Browser)                         │
│  ┌────────────┐  ┌─────────────┐  ┌──────────────────────────┐  │
│  │ Three.js   │  │ WebGL       │  │ GLSL Shaders             │  │
│  │ Scene Mgr  │──│ Renderer    │──│ (Splat / Post-Process)   │  │
│  └──────┬─────┘  └──────┬──────┘  └──────────────────────────┘  │
│         │               │                                       │
│  ┌──────┴───────────────┴──────────────────────────────────┐    │
│  │           Adaptive LOD Frustum Culling Engine            │    │
│  │              O(N log N) Spatial Tiling                   │    │
│  └─────────────────────┬───────────────────────────────────┘    │
└────────────────────────┼────────────────────────────────────────┘
                         │ Binary Splat Tiles (HTTP)
┌────────────────────────┼────────────────────────────────────────┐
│                    SERVER (FastAPI)                              │
│  ┌─────────────────────┴───────────────────────────────────┐    │
│  │              SplatTransformer Middleware                  │    │
│  │         (In-Memory 32-byte Binary Mutation)               │    │
│  └─────────────────────┬───────────────────────────────────┘    │
│                        │                                        │
│  ┌─────────┐    ┌──────┴──────┐    ┌──────────┐                │
│  │ Qdrant  │    │   Spatial   │    │  Neo4j   │                │
│  │ Vector  │◄──►│     RAG     │◄──►│  Graph   │                │
│  │   DB    │    │  Pipeline   │    │    DB    │                │
│  └─────────┘    └──────┬──────┘    └──────────┘                │
│                        │                                        │
│                 ┌──────┴──────┐                                 │
│                 │   LLM API   │                                 │
│                 │ (Llama 3 /  │                                 │
│                 │  GPT-4o)    │                                 │
│                 └─────────────┘                                 │
└─────────────────────────────────────────────────────────────────┘

---

📁 Project Structure

SpatialMind-AI/
├── .github/
│   └── workflows/
│       └── ci.yml              # Automated CI (lint + tests)
├── docs/
│   └── architecture.md         # Detailed architecture docs
├── frontend/
│   └── ...                     # Three.js WebGL client
├── src/
│   ├── __init__.py
│   ├── server.py               # FastAPI application server
│   ├── build_tiles.py          # Spatial tiling pipeline
│   ├── convert_new.py          # Splat format converter
│   ├── make_modes.py           # Multi-spectral vision modes
│   ├── ingest.py               # Data ingestion pipeline
│   └── start_workers.py        # Worker process manager
├── tests/
│   ├── test_phase3.py          # Phase 3 integration tests
│   └── test_transform.py       # Transform unit tests
├── index.html                  # Main WebGL application
├── landing.html                # Landing page
├── docker-compose.yml          # One-command environment setup
├── requirements.txt            # Python dependencies
├── scene_meta.json             # Scene configuration
├── .gitignore
├── LICENSE
└── README.md

---

🚀 Quick Start

Prerequisites

• Python 3.10+
• Docker & Docker Compose (for Qdrant + Neo4j)
• Node.js (optional, for frontend dev server)

1. Clone & Install

git clone https://github.com/Vapa1510/SpatialMind-AI.git
cd SpatialMind-AI
pip install -r requirements.txt

2. Start Infrastructure

docker-compose up -d    # Starts Qdrant + Neo4j

3. Run the Server

python server.py

4. Open the Viewer

Navigate to http://localhost:8000 in a WebGL-capable browser.

---

🧪 Running Tests

python -m pytest tests/ -v

---

🔧 Vision Modes

Mode	Description
🛰️ Satellite	Simulated overhead orthographic satellite imagery
📡 LiDAR	Point cloud visualization with intensity coloring
🌡️ Thermal IR	Infrared heat-map overlay on 3D scene
☢️ X-Ray	Wireframe transparency with depth-based opacity

---

🤝 Contributing

Contributions are welcome! Please open an issue first to discuss proposed changes.

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit changes (git commit -m 'Add amazing feature')
4. Push to branch (git push origin feature/amazing-feature)
5. Open a Pull Request

---

📄 License

This project is licensed under the MIT License — see the LICENSE file for details.

---

_{Built with ❤️ by Vansh}