Image Rendering

SharpConsoleUI renders pixel-based images in the console with automatic backend selection: full-resolution display via the Kitty graphics protocol in supported terminals (Kitty, WezTerm, Ghostty), with transparent fallback to Unicode half-block characters everywhere else.

Table of Contents#

1. Overview
2. PixelBuffer
3. Loading Images from Files
4. ImageControl
5. Scale Modes
6. Alignment and Scale Mode Interaction
7. Kitty Graphics Protocol
8. Half-Block Rendering
9. Creating Test Images

Overview#

The imaging system consists of:

• PixelBuffer — A 2D buffer of RGB pixels
• ImageControl — A BaseControl that displays a PixelBuffer with automatic rendering backend selection
• IImageRenderer — Strategy interface with two implementations:
  • KittyImageRenderer — Full-resolution rendering via the Kitty graphics protocol (virtual placements)
  • HalfBlockImageRenderer — Universal fallback using ▀ (U+2580), 2 pixels per cell
• ImageScaleMode — Controls how images scale to fit available space

The rendering backend is selected automatically at runtime based on terminal capabilities. No code changes are needed — the same ImageControl API works everywhere.

PixelBuffer#

A simple 2D pixel buffer for storing RGB image data.

// Create a 100x50 pixel buffer
var pixels = new PixelBuffer(100, 50);

// Set individual pixels
pixels.SetPixel(0, 0, new ImagePixel(255, 0, 0));   // Red
pixels.SetPixel(1, 0, new ImagePixel(0, 255, 0));   // Green
pixels.SetPixel(0, 1, new ImagePixel(0, 0, 255));   // Blue

// Get pixel
ImagePixel pixel = pixels.GetPixel(0, 0);

// Resize with bilinear interpolation
PixelBuffer resized = pixels.Resize(50, 25);

Creating from Arrays#

// From ImagePixel array (row-major order)
var pixelArray = new ImagePixel[width * height];
// ... fill array ...
var buffer = PixelBuffer.FromPixelArray(pixelArray, width, height);

// From ARGB int array (alpha is ignored)
var argbArray = new int[width * height];
// ... fill array ...
var buffer = PixelBuffer.FromArgbArray(argbArray, width, height);

Loading Images from Files#

SharpConsoleUI can load real image files using StbImageSharp. Supported formats: PNG, JPEG, BMP, GIF, TGA, identified by content rather than by file extension.

TIFF is not decodable at all. TIFF decoders have carried two heap out-of-bounds writes and a header that never terminates, and images often arrive from somewhere the application does not control, so no TIFF decoder is linked into the library. A TIFF, like any other unsupported input, raises UnsupportedImageFormatException.

From a File Path#

var buffer = PixelBuffer.FromFile("photo.png");
window.AddControl(Controls.Image(buffer));

From a Stream#

using var stream = File.OpenRead("photo.jpg");
var buffer = PixelBuffer.FromStream(stream);

From Another Imaging Library#

If another library already produced the pixels — because you need a format this one does not decode, or you applied processing of your own — copy them into a byte buffer and hand them over. No imaging dependency is shared with SharpConsoleUI.

// Tightly packed 24-bit RGB, row-major (3 bytes per pixel).
var buffer = PixelBuffer.FromRgb24Bytes(rgbBytes, width, height);

// Or 32-bit RGBA, where the alpha byte is ignored.
var buffer = PixelBuffer.FromRgba32Bytes(rgbaBytes, width, height);

With ImageSharp, for example:

using var image = Image.Load<Rgb24>("photo.webp");
var pixels = new byte[image.Width * image.Height * 3];
image.CopyPixelDataTo(pixels);
var buffer = PixelBuffer.FromRgb24Bytes(pixels, image.Width, image.Height);

With File Picker Dialog#

var path = await FileDialogs.ShowFilePickerAsync(windowSystem,
    filter: "*.png;*.jpg;*.jpeg;*.bmp;*.gif;*.tga");

if (path != null)
{
    var buffer = PixelBuffer.FromFile(path);
    imageControl.Source = buffer;
}

ImageControl#

Add images to windows using ImageControl:

// Create and add image
var pixels = new PixelBuffer(40, 20);
// ... fill pixels ...

window.AddControl(new ImageControl
{
    Source = pixels,
    ScaleMode = ImageScaleMode.Fit
});

// Via builder
builder.AddControl(Controls.Image(pixels));

Properties#

Setting Source or ScaleMode automatically invalidates the render cache and triggers a repaint.

Builder API#

Controls.Image(...) returns an ImageControlBuilder. The PixelBuffer overload used above is a shorthand that builds the control directly; use the builder when you need layout or identity options.

var image = Controls.Image()
    .WithSource(pixels)
    .WithScaleMode(ImageScaleMode.Fill)
    .WithAlignment(HorizontalAlignment.Center)
    .WithMargin(1)
    .WithName("preview")
    .Build();

Scale Modes#

public enum ImageScaleMode
{
    Fit,     // Scale to fit within bounds, preserving aspect ratio (no upscale)
    Fill,    // Scale to cover bounds, cropping as needed
    Stretch, // Stretch to fill bounds exactly, ignoring aspect ratio
    None     // Display at natural pixel size, clipped to bounds
}

Behavior Summary#

Examples#

Given a 20x10 pixel image (natural: 20 cols x 5 rows) in a 40x20 space:

• Fit: 20x5 (natural size, doesn't upscale)
• Fill: 40x20 (scales up to cover, may crop)
• Stretch: 40x20 (distorts to fill exactly)
• None: 20x5 (natural size, no scaling)

Alignment and Scale Mode Interaction#

Alignment and scale mode are independent concerns:

• Alignment (HorizontalAlignment, VerticalAlignment) — How much space the control claims from the layout
• ScaleMode — How the image fits within the control's allocated space

Without Explicit Alignment (defaults: Left/Top)#

• Fit and None use natural image dimensions
• Fill and Stretch expand to use available constraint space

// Fit: claims natural size (20x5)
new ImageControl { Source = pixels, ScaleMode = ImageScaleMode.Fit };

// Stretch: claims full constraint space (e.g., 80x25)
new ImageControl { Source = pixels, ScaleMode = ImageScaleMode.Stretch };

With Explicit Alignment#

Setting HorizontalAlignment.Stretch or VerticalAlignment.Fill forces the control to claim all available space in that dimension, regardless of scale mode.

// Fit with stretch alignment: claims full width/height,
// but image is scaled to fit within (preserving aspect ratio)
new ImageControl
{
    Source = pixels,
    ScaleMode = ImageScaleMode.Fit,
    HorizontalAlignment = HorizontalAlignment.Stretch,
    VerticalAlignment = VerticalAlignment.Fill
};

Kitty Graphics Protocol#

In terminals that support the Kitty graphics protocol (Kitty, WezTerm, Ghostty), images are rendered at full pixel resolution using virtual placements. This produces dramatically sharper results compared to half-block rendering.

How It Works#

1. Detection — At startup, TerminalCapabilities.Probe() sends a Kitty graphics query. If the terminal responds, or if KITTY_PID/WEZTERM_PANE environment variables are set, Kitty support is enabled.

2. Async PNG encoding — When an image source is set, the PixelBuffer is encoded to PNG on a background thread. A centered "Loading..." placeholder is shown while encoding completes. This keeps the UI responsive for large images.

3. Transmission — The PNG is transmitted to the terminal via APC escape sequences with U=1 (virtual placement mode). The image is assigned a unique ID and sized to span the target cell area (c columns, r rows).

4. Virtual placements — Each cell in the image area receives a U+10EEEE placeholder character with combining diacritics encoding the row and column. The terminal replaces these placeholders with the corresponding image pixels.

5. Resize optimization — The PNG is cached per source. When the control resizes, only the terminal placement is updated (delete + retransmit with new dimensions). No re-encoding occurs, making resize nearly instant.

Supported Terminals#

Architecture#

ImageControl.PaintDOM()
  |
  +-- ResolveRenderer() (once, on first paint)
  |     +-- Kitty detected? --> KittyImageRenderer
  |     +-- Otherwise      --> HalfBlockImageRenderer
  |
  +-- renderer.Paint(buffer, ...)
        +-- KittyImageRenderer:
        |     1. Encode PNG async (first time / source change)
        |     2. Transmit via IGraphicsProtocol
        |     3. Write U+10EEEE placeholder cells
        |
        +-- HalfBlockImageRenderer:
              1. Render half-block cells (existing behavior)

Half-Block Rendering#

The HalfBlockRenderer converts pixel data to console cells using the ▀ (upper half block, U+2580) character:

For each cell at (x, y):
  - Foreground color = top pixel  (image row y*2)
  - Background color = bottom pixel (image row y*2+1)
  - Character = '▀'

This gives 2x vertical resolution compared to using full characters.

For odd-height images, the last row uses ▀ with the background set to the window's background color.

Direct Usage#

// Render at natural size
Cell[,] cells = HalfBlockRenderer.Render(pixelBuffer, backgroundColor);

// Render at specific dimensions (with bilinear resize)
Cell[,] cells = HalfBlockRenderer.RenderScaled(
    pixelBuffer, targetCols, targetRows, backgroundColor);

Creating Test Images#

Generate images programmatically without file dependencies:

// Rainbow gradient
var pixels = new PixelBuffer(80, 40);
for (int y = 0; y < 40; y++)
{
    for (int x = 0; x < 80; x++)
    {
        double hue = (double)x / 80 * 360;
        var color = HsvToRgb(hue, 1.0, 1.0);
        pixels.SetPixel(x, y, color);
    }
}

// Checkerboard
var checker = new PixelBuffer(40, 40);
for (int y = 0; y < 40; y++)
{
    for (int x = 0; x < 40; x++)
    {
        bool isWhite = (x / 4 + y / 4) % 2 == 0;
        checker.SetPixel(x, y, isWhite
            ? new ImagePixel(255, 255, 255)
            : new ImagePixel(0, 0, 0));
    }
}

Configuration#

Image defaults are in ImagingDefaults:

See Also#

• Controls Reference — All available controls
• DOM Layout System — How controls are measured and arranged
• Compositor Effects — Post-processing effects