Added a detailed specification for CDN functionality, including: - Lightweight CDN node concept (simple executable for users) - Secure connectivity using pre-shared keys - Dynamic registration with main metadata server and DNS assignment - Content awareness, replication, and streaming fallback - Peer-to-peer (P2P) sharing for distributed content transfer - Federation and collaboration between CDNs - Future idea: predictive caching based on user activity This formalizes the CDN architecture for Walros, supporting distributed caching, federation, and smarter prefetching mechanisms."
6.4 KiB
6.4 KiB
Core Features
-
Media Playback: Support on-demand playback for media files in various formats.
- Caching & Performance:
- Use cached segments for seamless seeking and fast resume.
- If another user starts playback with the same parameters, reuse existing cached data to minimize transcoding and bandwidth usage.
- Caching & Performance:
-
CDN Capabilities and P2P Federation
-
Lightweight CDN Node:
- Each CDN node runs as a standalone executable binary that can be easily deployed by both technical and non-technical users.
- Minimal configuration required:
- Data Directory: Defines where cached media files are stored.
- Optional Name: Human-readable CDN identifier (used for UI labeling and dynamic DNS registration).
- Mode: Can operate as a shared CDN (available to multiple users) or a local-only CDN.
-
Secure Connectivity:
- Each CDN connects to the main metadata server using a pre-shared key for authentication and trust establishment.
- This ensures only authorized CDNs can register and exchange data with the main network.
-
Dynamic Registration & Discovery:
- Upon startup, the CDN connects to the main metadata server and automatically registers itself.
- The main server assigns a dynamic DNS name to each CDN for seamless connectivity.
- CDNs periodically update their metadata to report available content, storage capacity, and health status.
-
Content Awareness & Distribution:
- The main server maintains a global index of all available CDNs and the content cached on each.
- Users can choose which CDN should serve their content in the client UI.
- If a requested content is missing from the selected CDN, users have two options:
- Request content replication: Copy or prefetch the content from another CDN or the main server.
- Stream fallback: Stream the content directly from the main server or the nearest available CDN.
-
Peer-to-Peer (P2P) Sharing:
- CDNs and the main server interconnect via P2P protocols to simplify file transfer and reduce reliance on centralized infrastructure.
- This enables faster, bandwidth-efficient distribution of popular content between geographically distributed CDNs.
-
Federation & Collaboration:
- Multiple CDNs can share content and metadata within a trusted federation network.
- The system can automatically balance or replicate popular content across CDNs to optimize bandwidth usage and ensure availability.
-
Future Idea - Predictive Caching:
- CDNs may automatically pre-cache upcoming content based on user activity patterns.
- If a user watches the first movie in a series, the CDN pre-fetches sequels (e.g., parts 2, 3, etc.).
- If a user starts a new TV show, the CDN preloads subsequent episodes or seasons.
- This predictive caching minimizes latency and improves the user experience by ensuring likely-next content is available instantly.
- CDNs may automatically pre-cache upcoming content based on user activity patterns.
-
-
Metadata Management:
- Automatically identify and fetch metadata from databases like IMDb, TVDB, etc.
- Allow users to edit and correct metadata manually.
-
Transcoding Options:
- Pre-encode media into multiple formats for reduced compute load (at the expense of storage).
- Or use dynamic, on-the-fly transcoding for flexibility and storage efficiency.
-
Clients
- Web Client
- Native Clients: Desktop (Windows, Linux, MacOS), Mobile (Android, iOS), TV (Android TV, Apple TV, WebOS, Roku, Someone could create one for Xbox)
- Multiuser Support in clients: You are able to login with multiple users and change between them.
-
Offline Access:
- Clients can batch-download content for offline playback.
-
Scalability:
- Support distributed and remote ffmpeg-based transcoding pipelines for high availability and load balancing.
-
User Management:
- Authentication, profiles, access control, and session tracking.
-
Library & Personalization:
- Organize and manage libraries by media type (movies, shows, etc.).
- Create and manage playlists and collections.
- Track user history, progress, and play counts.
- Allow users to group media into watchlists, favorites, or custom categories.
- Provide statistics and usage insights (e.g., most watched, time spent, active users).
- Show statics for users about themselves
- Resume from last position across devices.
Nice to Have / Later
- Sync Play: Watch movies or shows together in real time across multiple clients.
- Integration with ARR Stack: Compatibility with Radarr, Sonarr, Lidarr, and similar automation tools.
- Parental controls
- Jellyfin-Compatible API and Plex-Compatible API
- Plugin Support: Extend functionality with custom or third-party plugins.
- Smart Playback Features:
- Intro, outro, recap skipper.
- Recommendations: Personalized movie and series suggestions based on user history and preferences.
- External Sync: Sync watch history, ratings, and metadata with services like Trakt.tv.
- Subtitle Management: Automatic subtitle fetching, editing, and syncing with playback. - Probably bazarr is enough for that.
- Multi-Format Detection: Detect multiple versions (e.g., 1080p, 4K, HDR) of the same media and show them in a unified view.
- IPTV / Live TV: Stream and manage live channels alongside on-demand content.
Architecture
- Frontend/Backend Separation: Clearly separated user interface and backend services.
- Microservice-Ready: Modular, loosely coupled components that can scale independently.
- Resiliency: Temporary outages in one service should not disrupt the viewing experience.
- Extensible Protocol Support: Easily add or replace streaming and media delivery protocols.
- Telemetry & Observability: Built-in tracing, logging, and metrics collection.
- Plugin System: Enable community and custom extensions.
- Robust API Design: Clean, well-documented REST and gRPC interfaces for interoperability.
Patrik's Dreams (Not Necessarily Realistic Goals)
- On-demand streaming over QUIC. (WebRTC was suggested but may not fit our use case.)
- Segmented Formats: Support CMAF alongside traditional MPEG-TS for more efficient adaptive streaming.
- Streaming Standards: Full compatibility with both HLS and MPEG-DASH.
- Processing Backends: Flexible support for ffmpeg, GStreamer, or alternative media frameworks.