Bálint Réthelyi (Blint)
6009407c99
Added a detailed specification for CDN functionality, including: - Lightweight CDN node concept (simple executable for users) - Secure connectivity using pre-shared keys - Dynamic registration with main metadata server and DNS assignment - Content awareness, replication, and streaming fallback - Peer-to-peer (P2P) sharing for distributed content transfer - Federation and collaboration between CDNs - Future idea: predictive caching based on user activity This formalizes the CDN architecture for Walros, supporting distributed caching, federation, and smarter prefetching mechanisms."
100 lines
6.4 KiB
Markdown
100 lines
6.4 KiB
Markdown
## Core Features
|
||
|
||
- **Media Playback:** Support on-demand playback for media files in various formats.
|
||
- **Caching & Performance:**
|
||
- Use cached segments for seamless seeking and fast resume.
|
||
- If another user starts playback with the same parameters, reuse existing cached data to minimize transcoding and bandwidth usage.
|
||
- **CDN Capabilities and P2P Federation**
|
||
- **Lightweight CDN Node:**
|
||
- Each CDN node runs as a standalone executable binary that can be easily deployed by both technical and non-technical users.
|
||
- Minimal configuration required:
|
||
- **Data Directory:** Defines where cached media files are stored.
|
||
- **Optional Name:** Human-readable CDN identifier (used for UI labeling and dynamic DNS registration).
|
||
- **Mode:** Can operate as a **shared CDN** (available to multiple users) or a **local-only CDN**.
|
||
|
||
- **Secure Connectivity:**
|
||
- Each CDN connects to the **main metadata server** using a **pre-shared key** for authentication and trust establishment.
|
||
- This ensures only authorized CDNs can register and exchange data with the main network.
|
||
|
||
- **Dynamic Registration & Discovery:**
|
||
- Upon startup, the CDN connects to the **main metadata server** and automatically registers itself.
|
||
- The main server assigns a **dynamic DNS name** to each CDN for seamless connectivity.
|
||
- CDNs periodically update their metadata to report available content, storage capacity, and health status.
|
||
|
||
- **Content Awareness & Distribution:**
|
||
- The main server maintains a global index of all available CDNs and the content cached on each.
|
||
- Users can choose which CDN should serve their content in the client UI.
|
||
- If a requested content is missing from the selected CDN, users have two options:
|
||
1. **Request content replication:** Copy or prefetch the content from another CDN or the main server.
|
||
2. **Stream fallback:** Stream the content directly from the main server or the nearest available CDN.
|
||
|
||
- **Peer-to-Peer (P2P) Sharing:**
|
||
- CDNs and the main server interconnect via **P2P protocols** to simplify file transfer and reduce reliance on centralized infrastructure.
|
||
- This enables faster, bandwidth-efficient distribution of popular content between geographically distributed CDNs.
|
||
|
||
- **Federation & Collaboration:**
|
||
- Multiple CDNs can share content and metadata within a trusted federation network.
|
||
- The system can automatically balance or replicate popular content across CDNs to optimize bandwidth usage and ensure availability.
|
||
|
||
- **Future Idea – Predictive Caching:**
|
||
- CDNs may automatically pre-cache upcoming content based on user activity patterns.
|
||
- If a user watches the first movie in a series, the CDN pre-fetches sequels (e.g., parts 2, 3, etc.).
|
||
- If a user starts a new TV show, the CDN preloads subsequent episodes or seasons.
|
||
- This predictive caching minimizes latency and improves the user experience by ensuring likely-next content is available instantly.
|
||
|
||
- **Metadata Management:**
|
||
- Automatically identify and fetch metadata from databases like IMDb, TVDB, etc.
|
||
- Allow users to edit and correct metadata manually.
|
||
- **Transcoding Options:**
|
||
- Pre-encode media into multiple formats for reduced compute load (at the expense of storage).
|
||
- Or use dynamic, on-the-fly transcoding for flexibility and storage efficiency.
|
||
- **Clients**
|
||
- Web Client
|
||
- Native Clients: Desktop (Windows, Linux, MacOS), Mobile (Android, iOS), TV (Android TV, Apple TV, WebOS, Roku, Someone could create one for Xbox)
|
||
- Multiuser Support in clients: You are able to login with multiple users and change between them.
|
||
- **Offline Access:**
|
||
- Clients can batch-download content for offline playback.
|
||
- **Scalability:**
|
||
- Support distributed and remote ffmpeg-based transcoding pipelines for high availability and load balancing.
|
||
- **User Management:**
|
||
- Authentication, profiles, access control, and session tracking.
|
||
- **Library & Personalization:**
|
||
- Organize and manage libraries by media type (movies, shows, etc.).
|
||
- Create and manage playlists and collections.
|
||
- Track user history, progress, and play counts.
|
||
- Allow users to group media into watchlists, favorites, or custom categories.
|
||
- Provide statistics and usage insights (e.g., most watched, time spent, active users).
|
||
- Show statics for users about themselves
|
||
- Resume from last position across devices.
|
||
|
||
## Nice to Have / Later
|
||
|
||
- **Sync Play:** Watch movies or shows together in real time across multiple clients.
|
||
- **Integration with ARR Stack:** Compatibility with Radarr, Sonarr, Lidarr, and similar automation tools.
|
||
- **Parental controls**
|
||
- **Jellyfin-Compatible API** and **Plex-Compatible API**
|
||
- **Plugin Support:** Extend functionality with custom or third-party plugins.
|
||
- **Smart Playback Features:**
|
||
- Intro, outro, recap skipper.
|
||
- **Recommendations:** Personalized movie and series suggestions based on user history and preferences.
|
||
- **External Sync:** Sync watch history, ratings, and metadata with services like Trakt.tv.
|
||
- **Subtitle Management:** Automatic subtitle fetching, editing, and syncing with playback. - Probably bazarr is enough for that.
|
||
- **Multi-Format Detection:** Detect multiple versions (e.g., 1080p, 4K, HDR) of the same media and show them in a unified view.
|
||
- **IPTV / Live TV:** Stream and manage live channels alongside on-demand content.
|
||
|
||
## Architecture
|
||
|
||
- **Frontend/Backend Separation:** Clearly separated user interface and backend services.
|
||
- **Microservice-Ready:** Modular, loosely coupled components that can scale independently.
|
||
- **Resiliency**: Temporary outages in one service should not disrupt the viewing experience.
|
||
- **Extensible Protocol Support:** Easily add or replace streaming and media delivery protocols.
|
||
- **Telemetry & Observability:** Built-in tracing, logging, and metrics collection.
|
||
- **Plugin System:** Enable community and custom extensions.
|
||
- **Robust API Design:** Clean, well-documented REST and gRPC interfaces for interoperability.
|
||
|
||
## Patrik's Dreams (Not Necessarily Realistic Goals)
|
||
|
||
- On-demand streaming over QUIC. *(WebRTC was suggested but may not fit our use case.)*
|
||
- **Segmented Formats:** Support CMAF alongside traditional MPEG-TS for more efficient adaptive streaming.
|
||
- **Streaming Standards:** Full compatibility with both HLS and MPEG-DASH.
|
||
- **Processing Backends:** Flexible support for ffmpeg, GStreamer, or alternative media frameworks.
|