Video Playback

SharpConsoleUI can play video files directly in the terminal using VideoControl. Frames are decoded via FFmpeg and rendered using three visual modes: half-block (best color), ASCII (density characters), and braille (highest spatial resolution).

Inspiration: The rendering approach is inspired by buddy, a Python terminal video player that pioneered half-block + braille + ASCII modes with FFmpeg frame decoding. VideoControl brings the same concept natively into .NET with SharpConsoleUI's compositing pipeline, pre-allocated buffers, dynamic resize, and overlay controls.

Table of Contents

  1. Overview
  2. Requirements
  3. Quick Start
  4. Streaming
  5. VideoControl
  6. Render Modes
  7. Builder API
  8. Playback Controls
  9. Overlay Status Bar
  10. Dynamic Resize
  11. Events
  12. Architecture
  13. Error Handling
  14. Performance Notes
  15. Sample Videos
  16. Complete Example

Overview

The video system consists of:

  • VideoControl — A BaseControl that plays video files with three render modes
  • VideoFrameReader — Manages the FFmpeg subprocess, piping raw RGB24 frames
  • VideoFrameRenderer — Converts raw pixel data to Cell[,] arrays for terminal display
  • VideoRenderMode — Enum selecting half-block, ASCII, or braille rendering
  • VideoPlaybackState — Enum tracking stopped, playing, or paused state
  • VideoDefaults — Configuration constants (FPS, timeouts, overlay timing, etc.)
  • VideoControlBuilder — Fluent builder for VideoControl

Requirements

FFmpeg must be installed and available on the system PATH:

# Linux
sudo apt install ffmpeg

# macOS
brew install ffmpeg

# Windows
winget install ffmpeg

If FFmpeg is not found, VideoControl displays a friendly error message inside the control area with installation instructions.

Quick Start

// Simplest usage: play a video file
var video = Controls.Video("video.mp4")
    .Fill()
    .Build();

var window = new WindowBuilder(windowSystem)
    .WithTitle("Video Player")
    .WithSize(80, 30)
    .Centered()
    .AddControl(video)
    .BuildAndShow();

video.Play();

// Cleanup on close
window.OnClosed += (_, _) =>
{
    video.Stop();
    video.Dispose();
};

Streaming

VideoControl accepts any source that FFmpeg understands — not just local files. Pass a URL to Source, Stream(), or the builder's WithSource():

// HTTP/HTTPS — remote video file
video.Stream("https://example.com/video.mp4");

// HLS — adaptive streaming playlist
video.Stream("https://live.example.com/stream/playlist.m3u8");

// RTSP — IP camera or security feed
video.Stream("rtsp://camera.local:554/live");

// RTMP — live stream
video.Stream("rtmp://live.twitch.tv/app/stream_key");

// FTP
video.Stream("ftp://server.local/videos/clip.mp4");

// Via builder
var video = Controls.Video("https://example.com/video.mp4")
    .Fill()
    .WithOverlay()
    .Build();

Streaming Notes

  • No seeking on live streams — Stream() starts from the current point; seek is automatically disabled when the source has no known duration
  • Duration unknown — DurationSeconds returns 0 for live streams; the overlay shows elapsed time only
  • Buffering — FFmpeg handles network buffering internally; frames arrive as they're decoded
  • Reconnection — if the stream drops, playback stops; call Stream() again to reconnect
  • Dynamic resize works with streams — FFmpeg restarts at the new resolution

VideoControl

Properties

Property Type Description
Source string? Video source — file path or URL (HTTP, RTSP, HLS, RTMP, FTP, etc.)
FilePath string? Alias for Source (backward compatibility)
RenderMode VideoRenderMode Current render mode (default: HalfBlock)
PlaybackState VideoPlaybackState Current state: Stopped, Playing, or Paused
TargetFps int Target frame rate, clamped 1–120 (default: 30)
Looping bool Whether playback loops (default: false)
OverlayEnabled bool Whether the overlay status bar is enabled
DurationSeconds double Total video duration in seconds (read-only)
CurrentTime double Current playback position in seconds (read-only)
FrameCount long Total frames rendered since play started (read-only)
ErrorMessage string? Error message shown in control (read-only)
IsEnabled bool Whether input is processed (default: true)

Methods

Method Description
Play() Starts or resumes playback
Pause() Pauses playback
TogglePlayPause() Toggles between play and pause
Stop() Stops playback and releases FFmpeg
CycleRenderMode() Cycles: HalfBlock → ASCII → Braille → HalfBlock
PlayFile(string path) Stops current, sets file path, starts playing
Stream(string url) Stops current, sets source URL, starts playing

Render Modes

Half-Block (Default)

Uses the Unicode upper half block character (▀, U+2580). Each terminal cell encodes 2 vertical pixels: the foreground color is the top pixel, the background color is the bottom pixel.

  • Resolution: 2x vertical pixel density
  • Color fidelity: Best — full 24-bit RGB per pixel
  • Best for: Color-rich video content
▀▀▀▀▀▀▀▀    ← Each cell = 2 pixels vertically
▀▀▀▀▀▀▀▀       fg = top pixel color
▀▀▀▀▀▀▀▀       bg = bottom pixel color

ASCII

Maps pixel brightness to a density character ramp: .:-=+*#%@ (10 levels from sparse to dense). Each cell is colored with the pixel's RGB as foreground.

  • Resolution: 1:1 with terminal grid
  • Color fidelity: Good — foreground colored, dark background
  • Best for: Retro aesthetic, lower bandwidth

Braille

Uses Unicode braille characters (U+2800–U+28FF). Each terminal cell covers a 2×4 pixel region (8 sub-pixels). Brightness is thresholded to activate dots; the cell's foreground color is the average RGB of the 8 pixels.

  • Resolution: Highest — 8 sub-pixels per cell
  • Color fidelity: Moderate — averaged colors
  • Best for: Maximizing spatial detail in small areas
Braille dot layout:     Bit values:
col0 col1               0x01  0x08
●    ●    row0          0x02  0x10
●    ●    row1          0x04  0x20
●    ●    row2          0x40  0x80
●    ●    row3

Switching Modes

// Set mode directly
videoControl.RenderMode = VideoRenderMode.Braille;

// Or cycle through modes
videoControl.CycleRenderMode();

Changing the render mode during playback automatically restarts FFmpeg with the correct pixel dimensions for the new mode, seeking to the current playback position.

Builder API

var video = Controls.Video()                // Create builder
    .WithSource("movie.mp4")                // File path, URL, or stream URI
    // .WithFile("movie.mp4")              // Alias — same as WithSource
    .WithRenderMode(VideoRenderMode.Ascii)  // Set render mode
    .WithTargetFps(24)                      // Set target FPS
    .WithLooping()                          // Enable looping
    .WithOverlay()                          // Enable overlay bar
    .Fill()                                 // Fill available space
    .WithMargin(1, 0, 1, 0)                // Set margins
    .WithName("mainVideo")                  // Set control name
    .OnPlaybackStateChanged((s, state) =>   // Subscribe to state changes
    {
        // state is VideoPlaybackState
    })
    .OnPlaybackEnded((s, e) =>              // Subscribe to playback end
    {
        // Video finished
    })
    .Build();

Shorthand with file path:

var video = Controls.Video("movie.mp4").Fill().Build();

Playback Controls

Keyboard

Key Action
Space Play / Pause toggle
M Cycle render mode
L Toggle looping
Esc Stop playback

Mouse

Clicking the video control gives it focus and shows the overlay status bar.

Programmatic

video.Play();
video.Pause();
video.TogglePlayPause();
video.Stop();
video.PlayFile("another.mp4");  // Stop + load + play
video.CycleRenderMode();
video.Looping = true;

Overlay Status Bar

When enabled via .WithOverlay(), a bottom-row overlay bar appears on user interaction (key press or mouse click) and auto-hides after 3 seconds.

The overlay shows:

  • Playback state icon: > (playing), || (paused), [] (stopped)
  • Current time / duration: 01:23 / 05:00
  • Render mode: HalfBlock, Ascii, or Braille
  • Loop indicator: Loop when enabled
  • Keyboard hints: Space:Play M:Mode L:Loop
// Enable overlay
var video = Controls.Video("movie.mp4")
    .WithOverlay()      // Enable
    .Build();

// Toggle at runtime
video.OverlayEnabled = true;
video.OverlayEnabled = false;

Dynamic Resize

When the window is resized during playback, VideoControl automatically:

  1. Detects the size change in PaintDOM (via DOM layout bounds)
  2. Saves the current playback timestamp
  3. Kills the current FFmpeg process
  4. Relaunches FFmpeg with the new pixel dimensions
  5. Seeks to the saved timestamp to continue seamlessly

This works in both Playing and Paused states.

Events

// Playback state changed (Playing, Paused, Stopped)
video.PlaybackStateChanged += (sender, state) =>
{
    Console.Title = $"State: {state}";
};

// Playback reached end of file
video.PlaybackEnded += (sender, args) =>
{
    // Cleanup or switch to next video
};

// Mouse events (from IMouseAwareControl)
video.MouseClick += (sender, args) => { /* ... */ };

Architecture

Frame Pipeline

┌──────────┐     raw RGB24     ┌──────────────────┐     Cell[,]     ┌──────────────┐
│  FFmpeg   │ ──────────────── │ VideoFrameRenderer│ ──────────────► │ CharacterBuffer│
│ subprocess│   byte[] pipe    │ (half/ascii/brail)│  pre-allocated  │  (PaintDOM)   │
└──────────┘                   └──────────────────┘                  └──────────────┘
     ▲                                                                      │
     │ stdin redirected (terminal safe)                                     ▼
     │                                                               ┌──────────┐
     └─── ffmpeg -i file -f rawvideo -pix_fmt rgb24 -s WxH -        │ Terminal │
                                                                     └──────────┘

Threading Model

  • UI thread: PaintDOM, property changes, overlay, resize detection
  • Background thread: PlaybackLoopAsync — frame reading, rendering, timing
  • Thread safety: _frameLock protects _currentFrameCells; Container?.Invalidate(true) is the only thread-safe call from background
  • UI marshaling: EnqueueOnUIThread for state changes from background (looping restart, error messages)

FFmpeg Integration

VideoControl shells out to the ffmpeg CLI rather than using a native binding:

  • Zero NuGet size impact — no bundled native libraries
  • Cross-platform — works wherever FFmpeg is installed
  • stdin redirected — prevents FFmpeg from corrupting terminal settings
  • stderr discarded — prevents pipe deadlock via BeginErrorReadLine
  • Process cleanup — Kill(entireProcessTree: true) on dispose

Error Handling

FFmpeg Not Found

When FFmpeg is not on PATH, the control displays a centered warning message:

FFmpeg not found. Install it to play videos:
  Linux:   sudo apt install ffmpeg
  macOS:   brew install ffmpeg
  Windows: winget install ffmpeg

File Not Found

A FileNotFoundException is thrown if the video path doesn't exist.

Corrupt Video / Decode Failure

If FFmpeg fails during decoding, the playback loop catches the exception and stops gracefully via EnqueueOnUIThread.

Performance Notes

  • Pre-allocated cell buffers: VideoFrameRenderer.RenderFrameInto() writes into a pre-allocated Cell[,] array — no per-frame GC allocation
  • Frame skipping: When rendering falls behind, up to 5 frames are skipped to catch up with wall-clock time
  • Pixel dimensions match render mode: FFmpeg scales to exactly the needed pixel count — half-block needs (cols, rows*2), ASCII needs (cols, rows), braille needs (cols*2, rows*4)
  • ~30 fps target: Configurable via TargetFps, capped at the video's native FPS

Sample Videos

The DemoApp includes two bundled sample videos:

File Duration Resolution Size Purpose
sample.mp4 4s 360p 322 KB Quick demo (colorful particles)
sample_bunny.mp4 30s 480p 4 MB Extended demo (Big Buck Bunny)

The Video Player demo falls back to sample.mp4 if the user cancels the file picker.

For automated testing, a deterministic test pattern is embedded as a resource:

  • SharpConsoleUI/Resources/test_sample.mp4 — 3s, 160×120, 10fps (49 KB)

Complete Example — Video Player App

A full, runnable console application that opens a file picker and plays the selected video. Copy this into a new project to get started immediately.

# Create a new project and add SharpConsoleUI
dotnet new console -n MyVideoPlayer
cd MyVideoPlayer
dotnet add package SharpConsoleUI

// Program.cs — Complete terminal video player

using SharpConsoleUI;
using SharpConsoleUI.Builders;
using SharpConsoleUI.Controls;
using SharpConsoleUI.Dialogs;
using SharpConsoleUI.Drivers;
using SharpConsoleUI.Helpers;
using SharpConsoleUI.Video;

// 1. Create the window system
//    RenderMode.Buffer enables double-buffered rendering for smooth output.
//    Hide the taskbar for a cleaner single-window app look.
var windowSystem = new ConsoleWindowSystem(
    RenderMode.Buffer,
    options: new ConsoleWindowSystemOptions(
        StatusBarOptions: new StatusBarOptions(ShowTaskBar: false)));

// 2. Set up status bar text — this appears at the top of the terminal
windowSystem.StatusBarStateService.TopStatus =
    "Video Player — Space: Play/Pause | M: Mode | L: Loop | Esc: Stop";

// 3. Handle Ctrl+C — shut down cleanly instead of hard-killing the process
Console.CancelKeyPress += (_, e) =>
{
    e.Cancel = true;
    windowSystem.Shutdown(0);
};

// 4. Build the VideoControl
//    Fill()        — stretch to use the entire window area
//    WithOverlay() — bottom status bar appears on key/click, hides after 3s
//    WithLooping() — restart from the beginning when the video ends
var videoControl = Controls.Video()
    .Fill()
    .WithOverlay()
    .WithLooping()
    .Build();

// 5. (Optional) React to playback state changes
videoControl.PlaybackStateChanged += (_, state) =>
{
    string mode = videoControl.RenderMode.ToString();
    string status = state switch
    {
        VideoPlaybackState.Playing => $"Playing ({mode})",
        VideoPlaybackState.Paused  => $"Paused ({mode})",
        _                          => "Stopped",
    };
    windowSystem.StatusBarStateService.TopStatus = $"Video Player — {status}";
};

// 6. Create the window and open a file picker asynchronously
//    WithAsyncWindowThread runs a background task tied to the window's lifetime.
//    The file picker is modal — it blocks this thread but not the UI.
//    BuildAndShow() creates the Window, registers it with the system, and displays it.
var window = new WindowBuilder(windowSystem)
    .WithTitle("Video Player")
    .Maximized()
    .WithColors(Color.White, Color.Black)
    .AddControl(videoControl)
    .WithAsyncWindowThread(async (win, ct) =>
    {
        // Open the file picker dialog
        var filePath = await FileDialogs.ShowFilePickerAsync(windowSystem,
            filter: "*.mp4;*.mkv;*.avi;*.webm;*.mov;*.flv;*.wmv");

        if (string.IsNullOrEmpty(filePath))
        {
            // User cancelled — exit the app
            windowSystem.EnqueueOnUIThread(() => windowSystem.Shutdown(0));
            return;
        }

        // Start playback on the UI thread
        // PlayFile() sets the path, launches FFmpeg, and begins decoding
        windowSystem.EnqueueOnUIThread(() =>
        {
            win.Title = $"Video — {Path.GetFileName(filePath)}";
            videoControl.PlayFile(filePath);
        });

        // Keep alive until the window closes (ct is cancelled)
        try { await Task.Delay(Timeout.Infinite, ct); }
        catch (OperationCanceledException) { }
    })
    .BuildAndShow();

// 7. Clean up when the window closes
//    Stop() cancels the playback loop; Dispose() kills the FFmpeg process.
window.OnClosed += (_, _) =>
{
    videoControl.Stop();
    videoControl.Dispose();
};

// 8. Run the window system — blocks until Shutdown() is called
await Task.Run(() => windowSystem.Run());

What this does

  1. Creates a maximized window with a black background — ideal for video
  2. Opens a file picker on launch to select a video file
  3. Plays the video with the overlay enabled — press any key to see playback info
  4. Updates the status bar in real-time with the current state and render mode
  5. Handles cleanup — FFmpeg process is killed when the window closes
  6. Exits cleanly on Ctrl+C or when the user cancels the file picker
Edit this page