stream-poc2

Very basic PoC to try out HLS with multiple audio and video tracks, with live transcoding using NVENC and scaling using CUDA via ffmpeg. (Subtitles are not supported) (Only 16:9 video is supported)

High-level overview

There is a "db" where all imported media data is stored.

The code is split into two: "import" and "serve".

"import" used to import various media files into that "db". During import, streams are extracted, and sliced up to be served by HLS. All audio streams are encoded to mp3 if needed, for better compatibility. Also, a few video slices are pre-encoded and stored in the "cache" folder.

"serve" runs the http server to watch the imported media and tune the various dials.

Those dials are:

• "Video" here are all the video "profiles" in the HLS stream, all of them except "orig" is transcoded live with ffmpeg and NVENC (if not served from the cache). (I didn't bother to implement any smartness, so formats "larger" than the source media is available). (If "orig" is not available here, that probably means an incompatible codec, or ffprobe could not figure out the bitrate of the source stream)
• "Audio" is the audio stream.
• "NVENC Preset" The various AVC/HEVC presets supported by NVENC this sets the value globally on the server

"profiles" are mostly just random numbers I punched in, see model/const.go for their details.

When tuning the dials, the video buffer is flushed, so the effect can be seen immanently. This is only for showcase as it is not desired in a real-world scenario.

Usage

Create a new folder for the "db":

This is where all metadata, original media chunks and cache is stored.

mkdir db

Import media files:

go run . import <path to media file> <media id>

"media id" is a url-safe string used to identify the media file on http and in the db folder.

Then start the server:

go run . server

Then open http://localhost:8080 in your browser, select the media and enjoy!

Code notes

chunks, slices and segments all mean the same trough the code.

db structure:

• db/ Root folder for the "db".
  • {id}/ Folder for each imported media, identified by "id".
    • meta.json All metadata used by the server to handle the video.
    • ffprobe.json Output of ffprobe stored for debugging purposes.
    • segments_video.csv Output of the segment data for the video stream. Used during import, kept for debugging.
    • segments_audio_{idx}.csv Output of the segment data for the audio stream identified by "idx". Used during import, kept for debugging.
    • video/ Folder for the video segments in the original format.
      • s{i}.ts Video segment number "i".
    • audio/ Folder for the audio segments in mp3 format.
      • {idx}/ Folder for the audio segments for the audio stream identified by "idx".
        • s{i}.ts Audio segment number "i".
    • cache/ Cache folder for pre-transcoded video segments.
      • video/ Video cache folder.
        • {profile}/ Folder for a specific transcoding profile.
          • s{i}.ts Transcoded video segment number "i".

Example for a HLS master/variant playlist

#EXTM3U

# Master playlist for bbb

#EXT-X-STREAM-INF:BANDWIDTH=70000,AVERAGE-BANDWIDTH=50000,CODECS="avc1.42E01E",RESOLUTION=176x144,FRAME-RATE=60.000000,AUDIO="audios",NAME="144p-avc",STABLE-VARIANT-ID"144p-avc"
video/144p-avc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=250000,AVERAGE-BANDWIDTH=200000,CODECS="avc1.42E01E",RESOLUTION=848x480,FRAME-RATE=60.000000,AUDIO="audios",NAME="480p-avc",STABLE-VARIANT-ID"480p-avc"
video/480p-avc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1000000,AVERAGE-BANDWIDTH=500000,CODECS="hvc1.1.6.L93.B0",RESOLUTION=1920x1080,FRAME-RATE=60.000000,AUDIO="audios",NAME="1080p-hevc",STABLE-VARIANT-ID"1080p-hevc"
video/1080p-hevc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=4000000,AVERAGE-BANDWIDTH=3500000,CODECS="hvc1.1.6.L93.B0",RESOLUTION=1920x1080,FRAME-RATE=60.000000,AUDIO="audios",NAME="1080p-hevc-hbr",STABLE-VARIANT-ID"1080p-hevc-hbr"
video/1080p-hevc-hbr/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=50000,AVERAGE-BANDWIDTH=40000,CODECS="hvc1.1.6.L93.B0",RESOLUTION=176x144,FRAME-RATE=60.000000,AUDIO="audios",NAME="144p-hevc",STABLE-VARIANT-ID"144p-hevc"
video/144p-hevc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=200000,AVERAGE-BANDWIDTH=170000,CODECS="hvc1.1.6.L93.B0",RESOLUTION=848x480,FRAME-RATE=60.000000,AUDIO="audios",NAME="480p-hevc",STABLE-VARIANT-ID"480p-hevc"
video/480p-hevc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=700000,AVERAGE-BANDWIDTH=500000,CODECS="avc1.42E01E",RESOLUTION=1280x720,FRAME-RATE=60.000000,AUDIO="audios",NAME="720p-avc",STABLE-VARIANT-ID"720p-avc"
video/720p-avc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=500000,AVERAGE-BANDWIDTH=400000,CODECS="hvc1.1.6.L93.B0",RESOLUTION=1280x720,FRAME-RATE=60.000000,AUDIO="audios",NAME="720p-hevc",STABLE-VARIANT-ID"720p-hevc"
video/720p-hevc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=1200000,AVERAGE-BANDWIDTH=700000,CODECS="avc1.42E01E",RESOLUTION=1920x1080,FRAME-RATE=60.000000,AUDIO="audios",NAME="1080p-avc",STABLE-VARIANT-ID"1080p-avc"
video/1080p-avc/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=4200000,AVERAGE-BANDWIDTH=3700000,CODECS="avc1.42E01E",RESOLUTION=1920x1080,FRAME-RATE=60.000000,AUDIO="audios",NAME="1080p-avc-hbr",STABLE-VARIANT-ID"1080p-avc-hbr"
video/1080p-avc-hbr/playlist.m3u8

#EXT-X-STREAM-INF:BANDWIDTH=4001453,AVERAGE-BANDWIDTH=4001453,CODECS="avc1.64632A",RESOLUTION=1920x1080,FRAME-RATE=60.000000,AUDIO="audios",NAME="orig",STABLE-VARIANT-ID"orig"
video/orig/playlist.m3u8

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audios",NAME="",LANGUAGE="und",AUTOSELECT=YES,DEFAULT=YES,CHANNELS="2",SAMPLE-RATE=48000,URI="audio/1/playlist.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audios",NAME="",LANGUAGE="und",CHANNELS="6",SAMPLE-RATE=48000,URI="audio/2/playlist.m3u8"

CODECS are just hard-coded defaults, they don't actually represent the real encoding parameters. hls.js seem to be fine with this.

Also CHANNELS="6" is fake as well, since the audio streams are transcoded to mp3 and mp3 can only have 2 channels.... hls.js seem to be fine with this.