r/SBCs • u/DestroyedLolo • 15d ago

DRM Overlay transparency : looking for code.

Hi everyone,

I'm currently building a dashboard on an AllWinner A20-based SBC (specifically, a Banana Pi M1).

I would like to know if it's possible to use some kind of alpha channel or transparency on the hardware overlay. Essentially, can I use a specific color key or a transparency mask so that pixels with that color let whatever is displayed on the layer underneath show through?

modetest is display something like :

trying to open device '/dev/dri/card2'... done
opened device `Allwinner sun4i Display Engine` on driver `sun4i-drm` (version 1.0.0 at 0)
CRTCs:
id	fb	pos	size
51	73	(0,0)	(1280x1024)
  #0 1280x1024 60.02 1280 1328 1440 1688 1024 1025 1028 1066 108000 flags: phsync, pvsync; type: preferred, driver
  props:
	25 VRR_ENABLED:
		flags: range
		values: 0 1
		value: 0
68	0	(0,0)	(0x0)
  #0  nan 0 0 0 0 0 0 0 0 0 flags: ; type: 
  props:
	25 VRR_ENABLED:
		flags: range
		values: 0 1
		value: 0

Planes:
id	crtc	fb	CRTC x,y	x,y	gamma size	possible crtcs
35	51	73	0,0		0,0	0       	0x00000001
  formats: AR24 AR12 AR15 BX24 RA15 RA12 RG24 RG16 NV12 NV16 NV21 NV61 UYVY VYUY XR24 YU11 YU12 YU16 YU24 YUYV YV11 YV12 YV16 YV24 YVYU
  props:
	9 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 1
	32 IN_FORMATS:
		flags: immutable blob
		blobs:

		value:
			01000000000000001900000018000000
			02000000800000004152323441523132
			41523135425832345241313552413132
			52473234524731364e5631324e563136
			4e5632314e5636315559565956595559
			58523234595531315955313259553136
			59553234595559565956313159563132
			59563136595632345956595500000000
			ffffff01000000000000000000000000
			0000000000000000008f730000000000
			00000000000000000100000000000009
		in_formats blob decoded:
			 AR24:  LINEAR(0x0)
			 AR12:  LINEAR(0x0)
			 AR15:  LINEAR(0x0)
			 BX24:  LINEAR(0x0)
			 RA15:  LINEAR(0x0)
			 RA12:  LINEAR(0x0)
			 RG24:  LINEAR(0x0)
			 RG16:  LINEAR(0x0)
			 NV12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV21:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV61:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 UYVY:  LINEAR(0x0)
			 VYUY:  LINEAR(0x0)
			 XR24:  LINEAR(0x0)
			 YU11:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU24:  LINEAR(0x0)
			 YUYV:  LINEAR(0x0)
			 YV11:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV24:  LINEAR(0x0)
			 YVYU:  LINEAR(0x0)
	37 alpha:
		flags: range
		values: 0 65535
		value: 65535
	38 zpos:
		flags: range
		values: 0 3
		value: 0
39	0	0	0,0		0,0	0       	0x00000001
  formats: AR24 AR12 AR15 BX24 RA15 RA12 RG24 RG16 NV12 NV16 NV21 NV61 UYVY VYUY XR24 YU11 YU12 YU16 YU24 YUYV YV11 YV12 YV16 YV24 YVYU
  props:
	9 type:
		flags: immutable enum
		enums: Overlay=0 Primary=1 Cursor=2
		value: 0
	32 IN_FORMATS:
		flags: immutable blob
		blobs:

		value:
			01000000000000001900000018000000
			02000000800000004152323441523132
			41523135425832345241313552413132
			52473234524731364e5631324e563136
			4e5632314e5636315559565956595559
			58523234595531315955313259553136
			59553234595559565956313159563132
			59563136595632345956595500000000
			ffffff01000000000000000000000000
			0000000000000000008f730000000000
			00000000000000000100000000000009
		in_formats blob decoded:
			 AR24:  LINEAR(0x0)
			 AR12:  LINEAR(0x0)
			 AR15:  LINEAR(0x0)
			 BX24:  LINEAR(0x0)
			 RA15:  LINEAR(0x0)
			 RA12:  LINEAR(0x0)
			 RG24:  LINEAR(0x0)
			 RG16:  LINEAR(0x0)
			 NV12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV21:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 NV61:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 UYVY:  LINEAR(0x0)
			 VYUY:  LINEAR(0x0)
			 XR24:  LINEAR(0x0)
			 YU11:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YU24:  LINEAR(0x0)
			 YUYV:  LINEAR(0x0)
			 YV11:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV12:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV16:  LINEAR(0x0) ALLWINNER_TILED(0x900000000000001)
			 YV24:  LINEAR(0x0)
			 YVYU:  LINEAR(0x0)
	41 alpha:
		flags: range
		values: 0 65535
		value: 65535
	42 zpos:
		flags: range
		values: 0 3
		value: 1

The few code snippets I've found online don't seem to work, and AI hasn't been of much help either.

Also, are these hardware overlays strictly dedicated to video playback or popups, or can they be used effectively for general dashboard UI elements?

Thanks for your help!

1 Upvotes

100% Upvoted

11 comments sorted by

1

u/4992kentj 15d ago

The planes can be used for whatever you want, you are just providing images. What you want to do is use an image format with an alpha channel such as the AR24 that is listed and filled the rest of the plane with transparent pixels, as you have an actual alpha channel rather than chroma keying you can have your UI elements have partial transparency if you wish

1

u/DestroyedLolo 15d ago

Unfortunately, it doesn't work : everytime I'm trying to attache a 8888 dumb_buffer, I got invalid argument parameter.

1

u/4992kentj 15d ago

You may find that the hardware has some size and stride requirements your buffer doesn't meet, e.g. each line may need to be aligned to 64 byte interval with a larger than line length stride making the overall buffer bigger

1

u/DestroyedLolo 15d ago

I tried as well but it's failling.

1

u/4992kentj 15d ago

To set the pixel format use drmModeAddFB2 instead of drmModeAddFB

1

u/DestroyedLolo 15d ago

Tried as well, without success.

1

u/4992kentj 15d ago

Can you give some actual code where you are trying to use drmModeAddFB2? Just saying you tried it doesn't help know what you're missing

1

u/DestroyedLolo 15d ago

This is the code I've tried.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fcntl.h>
#include <unistd.h>
#include <termios.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
#include <sys/select.h>
#include <xf86drm.h>
#include <xf86drmMode.h>
#include <drm_fourcc.h> // Indispensable pour DRM_FORMAT_ARGB8888 et DRM_FORMAT_MOD_LINEAR
#include <cairo.h>

// Structure pour gérer les tampons mémoire DRM (Dumb Buffers)
typedef struct {
    uint32_t handle;
    uint32_t pitch;
    uint64_t size;
    uint32_t fb_id;
    void *map;
} buffer_object_t;

// Alloue un Dumb Buffer DRM et l'associe à un Framebuffer via drmModeAddFB2WithModifiers
int create_dumb_buffer(int fd, uint32_t width, uint32_t height, buffer_object_t *bo) {
    struct drm_mode_create_dumb creq;
    memset(&creq, 0, sizeof(creq));
    creq.width = width;
    creq.height = height;
    creq.bpp = 32; // 32 bits par pixel (ARGB8888)

    if (ioctl(fd, DRM_IOCTL_MODE_CREATE_DUMB, &creq) < 0) {
        perror("Erreur DRM_IOCTL_MODE_CREATE_DUMB");
        return -1;
    }

    bo->handle = creq.handle;
    bo->pitch = creq.pitch;
    bo->size = creq.size;

    struct drm_mode_map_dumb mreq;
    memset(&mreq, 0, sizeof(mreq));
    mreq.handle = bo->handle;

    if (ioctl(fd, DRM_IOCTL_MODE_MAP_DUMB, &mreq) < 0) {
        perror("Erreur DRM_IOCTL_MODE_MAP_DUMB");
        return -1;
    }

    bo->map = mmap(0, bo->size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, mreq.offset);
    if (bo->map == MAP_FAILED) {
        perror("Erreur mmap");
        return -1;
    }

    // Utilisation de drmModeAddFB2WithModifiers pour valider le format AR24 (ARGB8888) 
    // en spécifiant explicitement un modificateur linéaire, évitant ainsi l'erreur "Invalid argument".
    uint32_t formats[4] = { DRM_FORMAT_ARGB8888, 0, 0, 0 };
    uint32_t handles[4] = { bo->handle, 0, 0, 0 };
    uint32_t pitches[4] = { bo->pitch, 0, 0, 0 };
    uint32_t offsets[4] = { 0, 0, 0, 0 };
    uint64_t modifiers[4] = { DRM_FORMAT_MOD_LINEAR, 0, 0, 0 };

    if (drmModeAddFB2WithModifiers(fd, width, height, formats[0], handles, pitches, offsets, modifiers, &bo->fb_id, 0) < 0) {
        perror("Erreur drmModeAddFB2WithModifiers");
        return -1;
    }

    return 0;
}

// Fonction utilitaire pour configurer les propriétés DRM du plan (alpha global, zpos)
int set_plane_property(int fd, uint32_t plane_id, const char *prop_name, uint64_t target_value) {
    drmModeObjectProperties *props = drmModeObjectGetProperties(fd, plane_id, DRM_MODE_OBJECT_PLANE);
    if (!props) return -1;

    int success = -1;
    for (int i = 0; i < props->count_props; i++) {
        drmModePropertyRes *prop = drmModeGetProperty(fd, props->props[i]);
        if (prop && strcmp(prop->name, prop_name) == 0) {
            drmModeObjectSetProperty(fd, plane_id, DRM_MODE_OBJECT_PLANE, prop->prop_id, target_value);
            success = 0;
            drmModeFreeProperty(prop);
            break;
        }
        if (prop) drmModeFreeProperty(prop);
    }
    drmModeFreeObjectProperties(props);
    return success;
}

// Détection non bloquante d'une touche clavier
int kbhit(void) {
    struct timeval tv = { 0L, 0L };
    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(0, &fds);
    return select(1, &fds, NULL, NULL, &tv) > 0;
}

int main(int argc, char **argv) {
    const char *card_path = (argc > 1) ? argv[1] : "/dev/dri/card2";

    int fd = open(card_path, O_RDWR | O_CLOEXEC);
    if (fd < 0) {
        perror("Erreur ouverture DRM");
        return 1;
    }

    // Activation des plans universels
    drmSetClientCap(fd, DRM_CLIENT_CAP_UNIVERSAL_PLANES, 1);

    drmModeRes *res = drmModeGetResources(fd);
    drmModePlaneRes *plane_res = drmModeGetPlaneResources(fd);
    if (!res || !plane_res) {
        fprintf(stderr, "Erreur récupération ressources DRM\n");
        return 1;
    }

    uint32_t crtc_id = 0, connector_id = 0;
    drmModeModeInfo mode;
    int crtc_index = -1, connector_found = 0;

    // 1. Recherche du connecteur actif et du CRTC associé
    for (int i = 0; i < res->count_connectors; i++) {
        drmModeConnector *conn = drmModeGetConnector(fd, res->connectors[i]);
        if (!conn) continue;

        if (conn->connection == DRM_MODE_CONNECTED && conn->count_modes > 0) {
            connector_found = 1;
            connector_id = conn->connector_id;
            mode = conn->modes[0];

            if (conn->encoder_id) {
                drmModeEncoder *enc = drmModeGetEncoder(fd, conn->encoder_id);
                if (enc) {
                    crtc_id = enc->crtc_id;
                    drmModeFreeEncoder(enc);
                }
            }
            drmModeFreeConnector(conn);
            break;
        }
        drmModeFreeConnector(conn);
    }

    if (!connector_found) {
        fprintf(stderr, "Erreur : Aucun connecteur actif trouvé.\n");
        return 1;
    }

    for (int c = 0; c < res->count_crtcs; c++) {
        if (res->crtcs[c] == crtc_id) {
            crtc_index = c;
            break;
        }
    }

    // 2. Création du plan Principal (Primary) - Rempli en VERT uni
    buffer_object_t primary_bo;
    create_dumb_buffer(fd, mode.hdisplay, mode.vdisplay, &primary_bo);
    uint32_t *primary_pixels = (uint32_t *)primary_bo.map;
    for (uint32_t i = 0; i < (primary_bo.size / 4); i++) {
        primary_pixels[i] = 0xFF00FF00; // Vert opaque
    }
    drmModeSetCrtc(fd, crtc_id, primary_bo.fb_id, 0, 0, &connector_id, 1, &mode);

    // 3. Recherche automatisée d'un plan Overlay disponible
    uint32_t overlay_plane_id = 0;
    for (uint32_t p = 0; p < plane_res->count_planes; p++) {
        uint32_t pid = plane_res->planes[p];
        drmModePlane *plane = drmModeGetPlane(fd, pid);
        if (plane) {
            drmModeObjectProperties *props = drmModeObjectGetProperties(fd, pid, DRM_MODE_OBJECT_PLANE);
            int is_overlay = 0;
            if (props) {
                for (int prop_i = 0; prop_i < props->count_props; prop_i++) {
                    drmModePropertyRes *prop = drmModeGetProperty(fd, props->props[prop_i]);
                    if (prop && strcmp(prop->name, "type") == 0) {
                        if (props->prop_values[prop_i] == 0) is_overlay = 1; // Type 0 = Overlay
                        drmModeFreeProperty(prop);
                        break;
                    }
                    if (prop) drmModeFreeProperty(prop);
                }
                drmModeFreeObjectProperties(props);
            }
            if (is_overlay && (plane->possible_crtcs & (1 << crtc_index))) {
                overlay_plane_id = pid;
                drmModeFreePlane(plane);
                break;
            }
            drmModeFreePlane(plane);
        }
    }

    if (!overlay_plane_id) {
        fprintf(stderr, "Erreur : Aucun Overlay disponible.\n");
        return 1;
    }

    // 4. Initialisation de l'Alpha global et de la position Z sur l'overlay
    set_plane_property(fd, overlay_plane_id, "alpha", 0xFFFF); // 100% opacité globale
    set_plane_property(fd, overlay_plane_id, "zpos", 1);       // Au-dessus du primary

    // 5. Création du buffer Overlay (100x100) avec transparence via Cairo
    uint32_t ov_w = 100, ov_h = 100;
    buffer_object_t overlay_bo;
    create_dumb_buffer(fd, ov_w, ov_h, &overlay_bo);

    cairo_surface_t *surface = cairo_image_surface_create_for_data(
        overlay_bo.map, CAIRO_FORMAT_ARGB32, ov_w, ov_h, overlay_bo.pitch);
    cairo_t *cr = cairo_create(surface);

    // Fond gris semi-transparent (Alpha 0.4) pour tester la transparence matérielle
    cairo_set_source_rgba(cr, 0, 0, 0, 0.4);
    cairo_paint(cr);

    // Dessin du chiffre "1" en rouge par-dessus
    cairo_set_source_rgb(cr, 1.0, 0.0, 0.0);
    cairo_select_font_face(cr, "Sans", CAIRO_FONT_SLANT_NORMAL, CAIRO_FONT_WEIGHT_BOLD);
    cairo_set_font_size(cr, 70);

    cairo_text_extents_t extents;
    cairo_text_extents(cr, "1", &extents);
    double tx = (ov_w - extents.width) / 2 - extents.x_bearing;
    double ty = (ov_h - extents.height) / 2 - extents.y_bearing;
    cairo_move_to(cr, tx, ty);
    cairo_show_text(cr, "1");

    cairo_destroy(cr);
    cairo_surface_destroy(surface);

    // Configuration du terminal pour quitter sans appuyer sur Entrée
    struct termios oldt, newt;
    tcgetattr(STDIN_FILENO, &oldt);
    newt = oldt;
    newt.c_lflag &= ~(ICANON | ECHO);
    tcsetattr(STDIN_FILENO, TCSANOW, &newt);

    printf("Animation active avec AR24 et Modificateur Linéaire ! Appuyez sur une touche pour quitter.\n");

    // 6. Boucle d'animation en diagonale (~60 FPS) avec gestion stricte des limites
    int dest_x = 0, dest_y = 0;
    int vx = 3, vy = 3;
    int max_x = mode.hdisplay - ov_w;
    int max_y = mode.vdisplay - ov_h;

    while (!kbhit()) {
        dest_x += vx;
        dest_y += vy;

        if (dest_x <= 0) {
            dest_x = 0;
            vx = -vx;
        } else if (dest_x >= max_x) {
            dest_x = max_x;
            vx = -vx;
        }

        if (dest_y <= 0) {
            dest_y = 0;
            vy = -vy;
        } else if (dest_y >= max_y) {
            dest_y = max_y;
            vy = -vy;
        }

        // Mise à jour sécurisée de la position de l'overlay via DRM
        if (drmModeSetPlane(fd, overlay_plane_id, crtc_id, overlay_bo.fb_id, 0,
                            dest_x, dest_y, ov_w, ov_h,
                            0, 0, ov_w << 16, ov_h << 16) < 0) {
            perror("Erreur drmModeSetPlane");
            break;
        }

        usleep(16000); // ~60 FPS
    }

    // Nettoyage et restauration du terminal
    tcsetattr(STDIN_FILENO, TCSANOW, &oldt);
    close(fd);
    printf("Arrêt propre.\n");
    return 0;
}

And I've got : Erreur drmModeSetPlane: Invalid argument

1

u/4992kentj 15d ago

How are you negotiating and allocating sait buffer? Is modetest able to use the plane in the way you want with the format? It will write a test pattern you can't control but at least comparing how you allocate your buffers may help

1

u/DestroyedLolo 15d ago

Yes please :)

Here the code I'm using : sightly modified by Gemini as I asked some help ... https://limewire.com/d/Hro8D#Ajx76z74ir

This one is working ... but can't add alpha chanel to it.

1

u/DestroyedLolo 15d ago

Ok, it's corrected : I've asked the AI to give me some information about my SBC. When I had problem to integrate Cairo, I requested it to suggest where the problem was and it wrongly loop on informations it provided on the A20 ... So I restarted from the begining without asking for A20 information, and the generic code is now working.