From 41fb1e5b55201b566def69e57c5b82cc89b5d6a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?B=C3=A1lint=20R=C3=A9thelyi=20=28Blint=29?= Date: Thu, 16 Oct 2025 09:15:21 +0200 Subject: [PATCH] docs: define CDN capabilities and P2P federation in goals.md Added a detailed specification for CDN functionality, including: - Lightweight CDN node concept (simple executable for users) - Secure connectivity using pre-shared keys - Dynamic registration with main metadata server and DNS assignment - Content awareness, replication, and streaming fallback - Peer-to-peer (P2P) sharing for distributed content transfer - Federation and collaboration between CDNs - Future idea: predictive caching based on user activity This formalizes the CDN architecture for Walros, supporting distributed caching, federation, and smarter prefetching mechanisms." --- goals.md | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-) diff --git a/goals.md b/goals.md index c3a60bb..a7703e0 100644 --- a/goals.md +++ b/goals.md @@ -4,8 +4,44 @@ - **Caching & Performance:** - Use cached segments for seamless seeking and fast resume. - If another user starts playback with the same parameters, reuse existing cached data to minimize transcoding and bandwidth usage. -- **CDN capabilities and P2P federation** - - TODO: Define this feature in detail. +- **CDN Capabilities and P2P Federation** + - **Lightweight CDN Node:** + - Each CDN node runs as a standalone executable binary that can be easily deployed by both technical and non-technical users. + - Minimal configuration required: + - **Data Directory:** Defines where cached media files are stored. + - **Optional Name:** Human-readable CDN identifier (used for UI labeling and dynamic DNS registration). + - **Mode:** Can operate as a **shared CDN** (available to multiple users) or a **local-only CDN**. + + - **Secure Connectivity:** + - Each CDN connects to the **main metadata server** using a **pre-shared key** for authentication and trust establishment. + - This ensures only authorized CDNs can register and exchange data with the main network. + + - **Dynamic Registration & Discovery:** + - Upon startup, the CDN connects to the **main metadata server** and automatically registers itself. + - The main server assigns a **dynamic DNS name** to each CDN for seamless connectivity. + - CDNs periodically update their metadata to report available content, storage capacity, and health status. + + - **Content Awareness & Distribution:** + - The main server maintains a global index of all available CDNs and the content cached on each. + - Users can choose which CDN should serve their content in the client UI. + - If a requested content is missing from the selected CDN, users have two options: + 1. **Request content replication:** Copy or prefetch the content from another CDN or the main server. + 2. **Stream fallback:** Stream the content directly from the main server or the nearest available CDN. + + - **Peer-to-Peer (P2P) Sharing:** + - CDNs and the main server interconnect via **P2P protocols** to simplify file transfer and reduce reliance on centralized infrastructure. + - This enables faster, bandwidth-efficient distribution of popular content between geographically distributed CDNs. + + - **Federation & Collaboration:** + - Multiple CDNs can share content and metadata within a trusted federation network. + - The system can automatically balance or replicate popular content across CDNs to optimize bandwidth usage and ensure availability. + + - **Future Idea - Predictive Caching:** + - CDNs may automatically pre-cache upcoming content based on user activity patterns. + - If a user watches the first movie in a series, the CDN pre-fetches sequels (e.g., parts 2, 3, etc.). + - If a user starts a new TV show, the CDN preloads subsequent episodes or seasons. + - This predictive caching minimizes latency and improves the user experience by ensuring likely-next content is available instantly. + - **Metadata Management:** - Automatically identify and fetch metadata from databases like IMDb, TVDB, etc. - Allow users to edit and correct metadata manually.