Development Log
A running record of what has been built, fixed, and shipped.
KUBERNETES CLUSTER SMOKE TESTS
Merged Kubernetes-native cluster smoke and durability test flow. Added RBAC configuration, test helper code, and Makefile wiring. Cluster integration tests now run entirely inside Kubernetes jobs with proper namespace isolation.
DURABLE RAFT WAL AND STABLE STORE
Merged durable Raft WAL and stable store implementation. Added CRC validation, envelope structure for corruption detection, and tests for log store and stable store corruption handling. Raft state now survives node restarts.
GRPC CLIENT RETRY AND TIMEOUT
Added client-side RPC timeout and improved gRPC client management with retry logic. Improved client routing robustness for leader changes and temporary node unavailability.
GRPC COMMUNICATOR SIMPLIFICATION
Reduced complexity in the gRPC communicator. Simplified client bootstrap logic and communicator internals. Removed accumulated state that was no longer needed after the smart client refactor.
SMART CLIENT AND TOPOLOGY ROUTER
Merged the smart client refactor. Introduced ConvergedRouter for automatic leader discovery and request routing. The client SDK now handles topology changes transparently without requiring manual leader tracking.
CONTROL PLANE ORCHESTRATOR REFACTOR
Refactored controlplaneorchestrator to include consensus message handling. Improved orchestrator interfaces to define the boundary for the new topology pattern.
CONVERGED ROUTER IMPLEMENTATION
Implemented ConvergedRouter for automatic request routing and topology management. The client SDK now handles topology changes transparently.
STANDARD REQUEST MANAGER
Added new SandstoreClient initialization and StandardRequestManager for robust client operation. Updated open smoke test according to new client library.
RAFT DATA PLANE ORCHESTRATOR
Implemented the RaftDataPlaneOrchestrator with ExecuteWrite and ExecuteRead methods. Updated ControlPlaneOrchestrator methods to include chunk location handling.
RAFT TRANSACTION COORDINATOR
Implemented the RaftTransactionCoordinator and RaftTxHandle with transaction management methods. Added a static endpoint resolver and legacy sorted placement strategy.
DOMAIN AND ORCHESTRATOR INTERFACES
Added domain and orchestrator interfaces defining the boundary for the new topology pattern. Introduced the decoupled architecture with ControlPlaneOrchestrator and DataPlaneOrchestrator.
DECOUPLED ORCHESTRATOR ARCHITECTURE
Fixed client chunk handling and integrated some transaction support in file services. Patched in the orchestrators and updated chunk handling to use ChunkDescriptor.
TRANSACTIONAL FILE SERVICE
Implemented TransactionalFileService for chunked file operations with two phase commit support. Refactored BoltMetadataService to support chunk intent operations and transaction handling.
LOCAL DISC CHUNK SERVICE UPDATES
Updated LocalDiscChunkService to support two phase commits and improved chunk management. Refactored Chunk and Metadata Services and set them up for two phase commit.
DURABILITY SMOKE TESTS
Implemented durability smoke tests and docker compose for localized testing. Moved config file to deploy docker directory.
DURABLE RAFT SNAPSHOTS
Implemented snapshot logic and wired durable raft into node. Refactored durable raft storage and added snapshot handling to ensure reliable replication.
RAFT METADATA REPLICATION COMPLETE
Fixed metadata duplication in leader node and completed Raft metadata replication system. The system now properly handles log entry processing for follower nodes and ensures consistent metadata across the cluster with proper quorum-based replication.
RAFT FILE SERVICE IMPLEMENTATION
Added a new Raft file service with strong dependency on Raft metadata replicator. Refactored store file functionality and improved quorum request handling for distributed file operations.
RAFT LEADER ELECTION IMPROVEMENTS
Fixed leader election race condition in Raft implementation and refactored metadata replicator interface to make it more modular. Enhanced the stability of the consensus algorithm.
RAFT METADATA LOG SYSTEM
Added MetadataLogEntry types and helper functions for metadata log operations. Implemented support for create and delete operations in the Raft log, with proper commit index management and log entry processing.
ENHANCED LOGGING SYSTEM
Improved local disc logging with levels and assigned integer levels to logging levels. Added better term logging for leader election processes to improve debugging and system monitoring.
MCP INTEGRATION COMPLETE
Merged the Model Context Protocol (MCP) feature branch. This integration allows AI systems to interact with Sandstore directly through a standardized interface. The MCP server now supports file operations including storing, reading, and deleting files.
CHUNK SERVICE IMPLEMENTATION
Added chunk service interface and file chunk struct. Renamed chunk service directory and implemented the foundation for distributed chunk storage with proper data structures.
GRPC COMMUNICATOR
Tested and added gRPC communicator implementation, providing an alternative to HTTP for inter-node communication. This gives the system flexibility in choosing communication protocols.
HTTP COMMUNICATOR IMPLEMENTATION
Developed the HTTP communicator component with functions for message handling, sending, and receiving. Added server functionality for message processing and implemented the core server structure.
PROJECT INCEPTION
Initialized the Sandstore project with the core message structure and communicator interface. The goal is to create a distributed file system that's easy to understand and modify for learning distributed systems concepts.