Covers
Let's show the albums's covers!!!
There are a few steps/questions to this:
Steps
- Generate nice 128x128 covers
- Index them properlly
- Display them!
Questions
- How to get small but still nice images?
- What image format to pick ? (POI or raw RGB?)
- Should I buffer them on the PSRAM ?
Generate the covers
Turns out the answer on how to get nice small is images is simple: Lanczos Resampling ! It is standard, I just never heard of it before, but it creates much better results on drastic scaledown than naive gaussian approaches.
Index the covers
Covers are by album, I did not have track_to_album indexing, so I first need it for arbitrary access ( I could get tracks from a playlist or from the general tracks/ index etc).
Also, some albums might not have a cover, so we need a fallback (basically the cover index 0, with some image)
Then I need to generate a big texture with all the images.
And finally I need a album_to_cover index and.... tadaaa
Display them
Now this is the trickiest part: I did not know if I should compress the images to minimize storage on the SD, bandwidth during transferts or if I should just store them as plain RGB.
So I did a benchmark:
main_cover_benchmark.cpp
cpp
// Standalone on-device benchmark: flash the cover_benchmark PlatformIO env
// (see platformio.ini) instead of the real firmware to compare QOI vs
// raw565 album-art loading on actual hardware. Lives in src/ (not
// test/onboard/, where the actual benchmark logic lives) purely because
// PlatformIO's build_src_filter can only select files within src_dir, and
// per-environment src_dir overrides aren't honored by this PlatformIO
// version (6.2.0) - see the comment on env:cover_benchmark in
// platformio.ini. build_src_filter excludes this file from the real
// nucleo_f446re env, and excludes main.cpp from this one, so exactly one
// setup()/loop() pair ever links.
//
// See .agents/covers-plan.md, Phase 0, and test/onboard/CoverBenchmark.h
// for what's measured.
//
// Before flashing: copy both indexer/generate-benchmark-covers.ts output
// folders (bench_qoi/, bench_raw565/) onto the SD card under /opentone/.
#include <Arduino.h>
#include <SdFat.h>
#include "pinout.h"
#include "rendering/ILI9341_GFX.h"
#include "storage/PSRAM_controller.hpp"
#include "../test/onboard/CoverBenchmark.h"
SdFat sd;
ILI9341_GFX display;
// CoverBenchmark holds ~73KB of buffers (COVER_BYTES_565 + MAX_QOI_BYTES,
// see CoverBenchmark.h) - global/static storage (.bss), not a local inside
// setup(). A local of this size would blow setup()'s stack frame the
// instant the function is entered (C++ reserves the full frame upfront,
// not lazily at each variable's declaration point), corrupting memory
// before any of its code even runs - this crashed psram.init() on real
// hardware even though the corruption's root cause was this line, not that
// function.
onboard_bench::CoverBenchmark bench(sd, display);
void setup() {
Serial.begin(115200);
while (!Serial)
delay(10);
Serial.println("=== Cover format benchmark (QOI vs raw565) ===");
// SD must be brought up before PSRAM - matches Audio_buffer::begin()'s
// order (src/sound/Audio_buffer.cpp), which is the only other place in
// this codebase that initializes both. Doing it the other way around
// hangs inside psram.init() on real hardware (confirmed on-device).
if (!sd.begin(SdSpiConfig(CARDCS, SHARED_SPI, SD_SCK_MHZ(25)))) {
Serial.println("ERROR: SD card init failed");
while (1) delay(1000);
}
Serial.println("SD ready");
if (!psram.init()) {
Serial.println("ERROR: PSRAM init failed");
while (1) delay(1000);
}
Serial.println("PSRAM ready");
display.begin();
display.fillScreen(0x0000);
Serial.println("Display ready");
delay(200);
auto qoiStats = bench.runSweep("QOI", "/opentone/bench_qoi/thumbs.bin",
"/opentone/bench_qoi/album_to_cover.bin");
bench.printStats("QOI", qoiStats);
delay(500);
auto raw565Stats =
bench.runSweep("raw565", "/opentone/bench_raw565/thumbs.bin",
"/opentone/bench_raw565/album_to_cover.bin");
bench.printStats("raw565", raw565Stats);
delay(500);
bench.benchmarkPsramCacheRoundtrip();
Serial.println("");
Serial.println("=== Benchmark complete ===");
}
void loop() {
delay(1000);
}Platormio.ini
[env:cover_benchmark]
platform = ststm32
board = nucleo_f446re
framework = arduino
debug_tool = stlink
monitor_speed = 115200
upload_protocol = stlink
build_src_filter = -<*> +<main_cover_benchmark.cpp> +<storage/PSRAM_controller.cpp>
build_flags =
-DUSE_UTF8_LONG_NAMES=1
lib_deps =
adafruit/Adafruit GFX Library@^1.11.9
adafruit/Adafruit BusIO@^1.15.0
arduino-libraries/SD@^1.3.0
greiman/SdFat@^2.3.0And some benchmark files
I got this:
=== Benchmark complete ===
=== Cover format benchmark (QOI vs raw565) ===
SD ready
=== PSRAM INIT START ===
PSRAM ID attempt 0: 0xD 0x5D
PSRAM OK
PSRAM ready
Display ready
loadCoverIndex /opentone/bench_qoi/album_to_cover.bin: fileSize=229 headerBytesRead=4 entryCount=25
entry 0: offset=0 length=36513 format=0
entry 1: offset=36513 length=37199 format=0
entry 2: offset=73712 length=28092 format=0
=== QOI sweep ===
album 0: read=27902us decode=5115us
album 1: read=28049us decode=5334us
album 2: read=21892us decode=4730us
album 3: read=2126us decode=781us
album 4: read=19365us decode=4603us
album 5: read=2722us decode=781us
album 6: read=2722us decode=781us
album 7: read=2721us decode=781us
album 8: read=2722us decode=781us
album 9: read=2723us decode=781us
album 10: read=2722us decode=781us
album 11: read=2722us decode=781us
album 12: read=2722us decode=781us
album 13: read=2725us decode=780us
album 14: read=2723us decode=781us
album 15: read=2724us decode=780us
album 16: read=2724us decode=781us
album 17: read=2722us decode=781us
album 18: read=2722us decode=781us
album 19: read=2724us decode=781us
album 20: read=2722us decode=781us
album 21: read=2723us decode=781us
album 22: read=2722us decode=781us
album 23: read=2722us decode=781us
album 24: read=2724us decode=780us
--- QOI summary ---
covers rendered: 25
avg read: 4736us max read: 26634us
avg decode: 1447us max decode: 5334us
avg total (read+decode): 6184us
loadCoverIndex /opentone/bench_raw565/album_to_cover.bin: fileSize=229 headerBytesRead=4 entryCount=25
entry 0: offset=0 length=32768 format=1
entry 1: offset=32768 length=32768 format=1
entry 2: offset=65536 length=32768 format=1
=== raw565 sweep ===
album 0: read=24764us decode=0us
album 1: read=24882us decode=0us
album 2: read=24888us decode=0us
album 3: read=24877us decode=0us
album 4: read=24878us decode=0us
album 5: read=24880us decode=0us
album 6: read=24878us decode=0us
album 7: read=24880us decode=0us
album 8: read=24879us decode=0us
album 9: read=24879us decode=0us
album 10: read=24880us decode=0us
album 11: read=24878us decode=0us
album 12: read=24881us decode=0us
album 13: read=24878us decode=0us
album 14: read=24881us decode=0us
album 15: read=24878us decode=0us
album 16: read=24879us decode=0us
album 17: read=24880us decode=0us
album 18: read=24879us decode=0us
album 19: read=24881us decode=0us
album 20: read=24879us decode=0us
album 21: read=24879us decode=0us
album 22: read=24880us decode=0us
album 23: read=24879us decode=0us
album 24: read=24881us decode=0us
--- raw565 summary ---
covers rendered: 25
avg read: 23460us max read: 23473us
avg decode: 0us max decode: 0us
avg total (read+decode): 23460us
=== PSRAM cover-cache roundtrip ===
PSRAM write (32KB): 77957us
PSRAM read (32KB): 78683us
=== Benchmark complete ===Conclusion
raw565 it is. much faser, less code, not much larger space on the disk.