Thumbnails and previews¶

The app shows pictures in three places: the small thumbnail in a file's row, the preview on the details page, and the fullscreen view. Everything that shows a file works from a URL: Coil (the image loader) and ExoPlayer (the video and audio player) both take one. This page explains where those URLs come from, how the login reaches them, and what the app reads out of files itself.

Paths below are under app/src/main/java/tools/senko/materialdrain/ unless they name a provider-* module.

The image loader¶

The app uses Coil 2.7 (io.coil-kt:coil-compose and coil-gif). There's one ImageLoader for the whole process, set up in AppContainer's init with Coil.setImageLoader. Every AsyncImage and context.imageLoader uses it.

coil.Coil.setImageLoader {
    coil.ImageLoader.Builder(application)
        .components {
            add(tools.senko.materialdrain.ui.media.MediaCovers.Factory(application))
            add(tools.senko.materialdrain.ui.media.AnimatedImages.Factory())
        }
        .okHttpClient { okHttpClient.newBuilder().readTimeout(30, java.util.concurrent.TimeUnit.SECONDS).build() }
        .build()
}

Part	What it adds
`MediaCovers.Factory`	A `Fetcher` for `audiocover://` URLs: a song's cover or a video's frame, read from the file. See Covers and frames
`AnimatedImages.Factory`	A `Decoder` for GIFs and animated WebP, so they move wherever they're shown
`okHttpClient`	The API's own client, so thumbnails reuse the HTTP/2 connection already open to the host. It gets a 30-second read timeout of its own: the API client has none (for long downloads), and a stalled thumbnail should give up

Animated images¶

AnimatedImages recognizes an animated file by its first bytes (GIF87a/GIF89a, or a WebP with a VP8X header whose animation flag is set), not by the type the host reports. Hosts often report none, or a generic one.

Android's ImageDecoder handles nearly all of them, but turns down some slightly broken GIFs that browsers still play. So, for files up to 64 MB, the decoder tries in turn: ImageDecoderDecoder, then Coil's older GifDecoder (for GIFs), then BitmapFactoryDecoder, which shows the first frame as a still picture. Bigger files go to ImageDecoder alone, which streams them instead of reading them whole first.

A private file's URL only works with the host's login. Coil and ExoPlayer don't know about hosts, so the app adds the headers itself. HostRequestAuth (in ui/media/MediaPlayerSupport.kt) holds one function, which AppContainer points at the active host:

HostRequestAuth.headersFor = { url ->
    providerRegistry.resolve(providerConfigStore.activeProviderId.value).requestHeaders(url)
}

Every place that loads a file asks it: imageRequest in ImagePreview.kt, the thumbnail in FileListItem, createMediaPlayer for ExoPlayer, and MediaCovers for the retriever. Each host's requestHeaders(url) only answers for its own private URLs and returns nothing for any other address, so a login never goes to a site it doesn't belong to. See Data and security.

S3 is the exception: it needs no headers, because its file URLs are presigned (the signature is in the URL itself).

Thumbnails¶

Which thumbnail a file gets¶

The file rows (browser/BrowserScreen.kt), the search results and the details page all ask FileInfoViewModel.thumbnailFor(node) which picture to show. The rules, in order:

SMB: whatever the host gives. An SMB share makes real thumbnails itself, on the device. See SMB files by address.
Audio: an audiocover://cover URL for the file's own album cover. A song without one keeps the app's music tile, not the host's generic picture.
Video: an audiocover://frame URL for a frame read from the file. The host's thumbnail is the fallback, for a video the device can't read either.
HEIC, HEIF and AVIF photos: the file itself (rawContentUrl). Hosts' thumbnailers often can't read these formats, so the phone decodes the file and Coil scales it down to the row's size.
Everything else: the host's thumbnail, but only for images and videos. For text, archives, apps and unknown kinds, hosts hand out a generic picture that says nothing. The app's own tile, coloured for the kind of file, says more.

// Only pictures and videos have a thumbnail worth showing: for everything else (text, archives, apps, unknown
// kinds) the hosts hand out a generic picture, and the app's own tile for the kind of file is used instead
val mime = node.previewMimeType().orEmpty()
return thumbnail.takeIf { mime.startsWith("image/") || mime.startsWith("video/") }

previewMimeType() uses the type the host reports, unless it's missing or application/octet-stream; then it works the type out from the file extension with MimeTypeMap.

The tile is always drawn first, and the thumbnail goes on top of it (FileIcon in ui/components/FileListItem.kt). There's no separate placeholder or error picture: if the thumbnail fails, the tile is simply still there.

What each host gives¶

This is thumbnailUrl(node) of each host, before the rules above are applied:

Host	`thumbnailUrl`
Pixeldrain	`/api/file/<id>/thumbnail` for a file with an id; `/api/filesystem/<path>?thumbnail` for a filesystem file
REST API	The `thumbnail_id` endpoint for a file with an id, or `browse_thumbnail` for a path. None without them
WebDAV	An image is its own thumbnail (the file's URL); Coil scales it down. Nothing for other files
S3	None
SMB	A `/thumb/` URL of `SmbContentServer`, for images, videos and audio

Covers and frames from the file¶

ui/media/MediaCovers.kt makes thumbnails for hosts that stream their files over HTTP(S). It builds URLs like this:

audiocover://cover?src=<file URL>&v=<modified date>
audiocover://frame?src=<file URL>&fallback=<host thumbnail>&v=<modified date>

v is the file's modified (or created) date, so a changed file gets a new thumbnail. Coil hands these URLs to MediaCovers.Factory, whose CoverFetcher:

Looks in its disk cache, keyed by the SHA-1 of the source URL and v. A cached .jpg is the thumbnail. A cached .none means the file was read before and has nothing.
Otherwise opens the file with MediaMetadataRetriever.setDataSource(url, headers). The media framework fetches only the byte ranges it needs, not the whole file. At most two files are read at once (a Semaphore(2)), because each takes a decoder and several requests.
For a song, takes the embeddedPicture. For a video, takes the key frame nearest to a tenth of the way in (past a fade-in or a title card), but never more than 30 seconds in. If that gives nothing, it tries the key frame before, then the very first frame.
Scales the picture to 320 px on its longest side and saves it as a JPEG.
With nothing to show, fetches the fallback (the host's thumbnail, with its login) through Coil's normal fetchers, or fails so the tile stays.

Only a file that was read and really has no picture is remembered as .none. A read that failed (the network, a slow host) is tried again next time.

Why read a frame when the host has a thumbnail?

Some hosts give a video they can't read (an unusual codec, an index at the end of the file) a generic picture of a film. The app can't tell that picture from a real thumbnail. Reading a frame on the device fixes most of them, and the host's picture is still the fallback.

Thumbnails are written as a .part file and then renamed, so one cut short by a crash is never read back.

Caching¶

Cache	Where	What
Coil's memory and disk caches	Coil's defaults	Every image Coil loads: thumbnails and previews. The app doesn't configure them
Song covers	`cacheDir/audio-covers`	Made by `MediaCovers`
Video frames	`cacheDir/video-frames-2`	Made by `MediaCovers`. A new folder: the first version also remembered videos that only failed to load once, and never tried them again
SMB thumbnails	`cacheDir/smb-thumbnails`	Made by `SmbContentServer`, kept between runs
Media streaming	`cacheDir/media_cache`	`ExoPlayerCache`: a `SimpleCache` of up to 100 MB, least recently used first out

All of them are in the app's cache folder, so Android can clear them when space runs low.

util/ExoPlayerCache.kt creates its SimpleCache once per process (a SimpleCache can't be opened twice on the same folder). createMediaPlayer wraps the OkHttp data source in a CacheDataSource with FLAG_IGNORE_CACHE_ON_ERROR, so a cache problem falls back to the network instead of failing playback.

Previews¶

The details page (files/FileDetailsScreen.kt) picks one preview by the file's previewMimeType():

The file is	Shown with	File
An archive (`zip`, `7z`, `rar`, `tar`, `tgz`, `apk`) on a host that can look inside	`ArchiveContents`	`files/ArchiveContents.kt`
`image/*`	`InlineImagePreview`	`ui/media/ImagePreview.kt`
`video/*`	`InlineVideoPreview`, then the fullscreen player	`ui/media/VideoPlayer.kt`
`audio/*`	`AudioPlayerPreview`, the song's card	`ui/media/AudioPlayer.kt`
A PDF (`application/pdf`, or a `.pdf` name)	`InlinePdfPreview`	`ui/media/PdfPreview.kt`
Markdown	`MarkdownPreview`	`ui/components/MarkdownPreview.kt`, parsed by `files/Markdown.kt`
Other text	`CodePreview`	`ui/components/CodePreview.kt`
Anything else with a thumbnail	The thumbnail in a card

The badge at a preview's corner opens it over the whole screen in FullScreenFileDialog (ui/media/FullScreenPreview.kt): one dialog for every kind, a page per file, in the app's dark colours whatever theme it's in. With Swipe between files, a swipe sideways goes to the next file of the same kind (FileGroup), which comes in like a page; a page the data saver held back loads as soon as it's shown, as going to it is asking for it.

Images¶

LayeredPreviewImage shows an image in two steps. The thumbnail (small, and usually cached already) appears right away, blurred, as a backdrop. The full image fades in over it once it has loaded, with a thin progress bar until then.

The thumbnail is always blurred, never shown as if it were the picture. Pixeldrain's thumbnails are made at a fixed size, so they're framed differently from the full image; blurred, they're just its colours, and nothing seems to jump when the full image arrives. Blurring needs Android 12; before that the backdrop is only made faint. The setting Blurred backdrop (AppSettings.blurredBackdrop) turns it off.

Fullscreen¶

A picture's page of FullScreenFileDialog (and FullScreenMediaPreviewDialog, the Upload screen's, which only has pictures and videos) puts it in a ZoomableBox: pinch to zoom, drag to pan, double-tap to zoom in and out, tap beside the picture to close. The image is decoded at most 3072 px on a side (maxPixels), so a huge photo doesn't fill the memory. Videos open FullscreenVideoPlayer.

Images use FilterQuality.None in the preview and fullscreen, so pixel art and zoomed-in pictures stay sharp instead of blurry.

Video and audio¶

Both players use ExoPlayer from Media3, made by createMediaPlayer in ui/media/MediaPlayerSupport.kt:

A remote file streams over the API's OkHttp client (one connection pool, HTTP/2), through the cache above, with the host's headers from HostRequestAuth. A local file (on the Upload screen) is read directly.
Playback starts after one second is buffered (the default waits for two and a half) and buffers up to 50 seconds ahead.
rememberManagedPlayer ties the player to the composable: it pauses when the app goes to the background and is released when the composable leaves.

AudioPlayerPreview is created with prepareNow = false, so nothing loads until you press play. A song you only glance at costs no data. The inline video preview is just the thumbnail with a play button; the player starts in fullscreen.

MediaControls.kt has the shared controls: MediaIconButton, MediaTextButton (e.g. the playback speed) and MediaSeekBar, which grows thicker while pressed, seeks on a tap and scrubs on a drag.

The video player (FullscreenVideoPlayer) takes its touches itself: a tap shows or hides the controls, a double tap skips back or on at the sides (by Skip by, SEEK_STEPS) and plays or pauses in the middle, holding it plays at 2× (HOLD_SPEED), a pinch zooms up to 4× (MAX_VIDEO_ZOOM). With Brightness and volume swipes, a swipe up or down changes the brightness on the left and the volume on the right, and sideways seeks; without them, a sideways swipe is left to the swipe between files. Its settings menu (VideoSettingsMenu) has Loop (one choice for all videos, AppSettings.loopVideos), Speed, Subtitles and Audio where the video has them, and Open in another app. The rotate button turns the activity, which doesn't restart for it (configChanges in the manifest), so the video goes on. Where each video was left is kept in util/VideoPositions.kt (the last 200, in preferences of their own, not backed up).

Songs in the background¶

With Play songs in the background, a song plays in PlaybackService (ui/media/PlaybackService.kt), a Media3 MediaSessionService that outlives the details and the app's screen, with its controls in the notification and on the lock screen. The details' song card reaches it through a MediaController (rememberPlaybackController); the item's id is the song's address.

The queue (SongQueue.kt): a song played from its details takes the songs of the list it was opened from, in its order, up to 100 on each side (QUEUE_REACH). Add to queue puts more at its end. BackgroundSong follows the player's song and queue for the screens, with its repeat mode and shuffle; the service shuffles anew each time shuffle is turned on, with the song playing first, and takes a queue rearranged while shuffled as a command of its own.
Tags (SongTags.kt): a song's title, artist and album, and what kind of file it is, are read out of the file the way its cover is (only the parts needed), a couple of songs at a time, and kept on disk. The songs around the one playing are read ahead, so a skip shows the next title at once.
Pictures: CoverBitmapLoader gives the notification and lock screen their covers through the app's own image loader, which reads what Media3's can't (audiocover://, a host's pictures behind its login); a song without one shows its file tile (rememberPlaceholderArt).
The miniplayer (MiniPlayer.kt, BackgroundMiniPlayer) floats above the navigation bar. It follows the finger: sideways skips (the next song peeking in beside it), down closes the song (closeSong, with an Undo where that's on), up pulls the player open. AudioPlayerView (AudioPlayerView.kt) is the player over the whole app, or the compact card the miniplayer grows into (CompactMorph moves the cover and title from the one's place to the other's); AudioPlayerSheetState keeps how far open it is, and survives the activity being made again.
Exported, but only so far: the service is exported, as the system's media controls need to reach it. Another app connected to it can play, pause, skip and seek, but not choose what plays, so it can't have the player load an address of its choosing.

Text¶

FileInfoViewModel.fetchTextFilePreviewContent decides whether a file is text: a text MIME type, an extension that SourceHighlighting.kt knows, or application/octet-stream with an extension like .log, .ini, .conf or .md. Then:

Limit	Value	Constant
Largest file that's fetched	1 MB	`MAX_TEXT_PREVIEW_FETCH_SIZE_BYTES`
Shown on the details page	The first 8 KB of text, cut at the last whole line	`MAX_TEXT_PREVIEW_DISPLAY_LENGTH`
Shown in fullscreen	Everything fetched

A bigger file shows "File is too large … for text preview" instead. The whole file is downloaded into memory through the same downloadNodeTo as a normal download, so it works for every host.

On the details page, CodePreview shows the start in a 16:9 card, the size of the other previews, scrolling inside, with what it is and how long ("JSON · 120 lines", "first 30 of 400 lines") over its bottom corner. While it loads it's the same card, so nothing below it moves when the text comes in. Its badge opens CodeFullscreenContent: numbered lines, wrapping or sideways scrolling (starting from the Wrap long lines setting), Find in the text (each match lit, stepped through with the arrows), bigger or smaller text, and copying all of it, or the lines picked with a long press on a line's number and a tap on another. Only the lines on screen are laid out, and the colouring is worked out off the main thread, so long files stay smooth; past MAX_COLOURED_LENGTH a text is shown without colours.

A Markdown file is shown as it's meant to look: parseMarkdown (files/Markdown.kt, plain Kotlin) turns it into blocks (headings, lists with tasks, quotes, code, tables), which MarkdownPreview draws in the same card, and full screen with a button to its source. Links to the web open in the browser.

The Upload screen has its own, simpler preview (InlineTextPreview in ui/media/TextPreview.kt): the first 4096 characters of the picked file.

Syntax highlighting¶

files/SourceHighlighting.kt is plain Kotlin, with no Android in it, so it's tested on the JVM (SourceHighlightingTest, RichHighlightingTest, LongInputHighlightTest). It has two parts:

The language list, SourceLanguages.byExtension: which extension is which language. This is the part to edit when a file type is missing. A file whose extension isn't listed can still be recognized by a #! first line (byShebang: python, bash, node, …).
The tokenizer, tokenize(code, language): finds comments, strings, numbers, keywords, types, calls, keys and so on. It's a plain scan, character by character with no regular expressions, so a very long input can't overflow the stack. It isn't a parser: a string simply ends at its quote or at the end of its line.

CodePreview.kt colours the tokens with one of two sets of colours, for light and dark backgrounds. The theme's own colours are too few and too close together to tell a type from a call from a key.

Archives¶

A file named .zip, .7z, .rar, .tar, .tgz or .apk is opened as an archive when the host has ArchiveOps (Pixeldrain, with its ?zip_info and ?zip_file API; a REST API config with archive_info and archive_file endpoints) and ArchiveOps.canOpen says it can reach this one (FileInfoViewModel.showsArchive): Pixeldrain only can for the files of its filesystem, not those of the account. The tab the details were opened from can also leave it out (ARCHIVE_BROWSE in its disabled_capabilities). The archive itself is never downloaded just to look inside, and the listing is read off the main thread.

ArchiveContents shows the folder you're in, its folders to open and its files to save, and a button to go up. Files inside can only be saved, not opened: downloadArchiveEntry starts an ordinary download, and downloadNodeTo reads the entry with ArchiveOps.read.

A host with no ArchiveOps of its own, whose files can be read in parts (WebDAV, S3, and SMB through the app's own server), gets the app's own for zips (DeviceZipArchives, files/DeviceArchives.kt; .zip, .cbz, .jar, .apk). It reads only the zip's directory at its end, in a few range requests, and for a file of it, only that file's part. Stored and deflated files are read, as nearly every zip has; a file with a password isn't. A host that doesn't answer for parts of a file can't be looked inside this way, which is said rather than the whole archive being downloaded.

PDFs¶

Android's PdfRenderer draws a PDF's pages, but only from a whole file on the device, so a PDF is loaded into the app's cache first (FileInfoViewModel.fetchPdfPreview, through the same downloadNodeTo as a download):

Limit	Value	Constant
Loaded without being asked for	16 MB; a bigger one waits for a tap (`PdfPreviewState.Large`)	`MAX_PDF_AUTO_LOAD_BYTES`
Largest that's loaded at all	64 MB, also when the host doesn't tell the size (`LimitedOutputStream` stops the copy)	`MAX_PDF_PREVIEW_BYTES`
Kept in `cacheDir/pdf-preview`	The last 5, by host, file, size and date, so a changed file isn't shown from an old copy	`PDF_CACHE_KEPT`

A held-back preview (the data saver) counts a tap on it as asking, whatever the size. A copy that left the cache while the details were open (the cache was cleared) is loaded again instead of being called damaged.

PdfPreview.kt opens the file with a PdfDocument, which reads every page's size once, so pages take their place before they're drawn, and draws one page at a time behind a Mutex (PdfRenderer can only have one page open). A page is drawn at most 2560 px wide and 8 million pixels, on white, as on paper. A damaged page with no width or height counts as square instead of crashing the layout.

In the details, InlinePdfPreview is a card like the other previews: the top of the first page, the number of pages, and a tap opens it full screen. While it loads it shows how far along it is.
Full screen, PdfFullscreenContent (a page of FullScreenFileDialog) shows every page in a LazyColumn, scrolled with a finger and zoomed with two (or a double tap, up to 5×). Zooming lays the pages out wider rather than stretching them, so scrolling keeps up with the finger, and each page is drawn again, sharp, once the zoom has settled (200 ms).
A password-protected PDF can't be opened by PdfRenderer; it says so instead.

SMB files by address¶

An SMB share has no web address for its files. But everything that shows a file in the app works from a URL: Coil for thumbnails and pictures, ExoPlayer for video and audio, MediaMetadataRetriever for covers. So the app gives each file an address itself, from a tiny HTTP server on the phone: SmbContentServer in provider-smb.

How it works¶

The server listens on 127.0.0.1 only, on a port Android picks (ServerSocket(0, 50, 127.0.0.1)). It starts the first time a URL is asked for. Each connection is handled on a thread from a pool.

Every URL starts with a token: 16 random bytes from SecureRandom, made fresh each time the app starts.

http://127.0.0.1:<port>/<token>/raw/<host id>/<path>
http://127.0.0.1:<port>/<token>/thumb/<host id>/<path>?v=<modified date>

Request	Answer
Not `GET` or `HEAD`	`405`
A wrong or missing token	`403`
An unknown host id	`404`
`/raw/…`	The file, with `Content-Type`, `Content-Length` and `Accept-Ranges: bytes`
`/raw/…` with `Range: bytes=…`	`206 Partial Content` with `Content-Range`, or `416` for a range outside the file
`/thumb/…`	A JPEG (or a PNG when the picture has see-through parts), with `Cache-Control: private, max-age=86400`

Range requests are what make a video seekable. When you drag the seek bar, ExoPlayer asks for the bytes from that point on, and the server reads them with SmbRandomAccessFile.seek instead of everything before. Only the first range of a multi-range request is served; players ask for one.

Each SmbStorageProvider registers its SmbClient under its host id, and its thumbnailUrl and rawContentUrl return these addresses.

Thumbnails on the device¶

For /thumb/, the server makes a thumbnail of up to 320 px from the file on the share:

Kind	Extensions	How
Image	`jpg`, `jpeg`, `png`, `webp`, `gif`, `bmp`, `heic`, `heif`, `avif`	Read whole and scaled down. Not for files over 40 MB, which would all have to come over the network
Video	`mp4`, `m4v`, `mkv`, `webm`, `mov`, `3gp`, `avi`, `ts`	A frame one second in, read with `MediaMetadataRetriever` through an `SmbMediaDataSource`, which reads only the parts it needs
Audio	`mp3`, `m4a`, `flac`, `ogg`, `opus`, `aac`, `wav`	The embedded cover

Images are decoded with ImageDecoder, which turns them the way their EXIF data says. Some HEIC photos from phone cameras are refused by ImageDecoder but read by BitmapFactory, so that's tried next. At most three thumbnails are made at once. Each is saved in cacheDir/smb-thumbnails (set by AppContainer), keyed by the SHA-1 of the host id, the path and v. A thumbnail that can't be made is not tried again until the app restarts.

Cleartext, only for 127.0.0.1¶

Android blocks plain http:// by default. app/src/main/res/xml/network_security_config.xml allows it for exactly one address:

<network-security-config>
    <domain-config cleartextTrafficPermitted="true">
        <domain includeSubdomains="false">127.0.0.1</domain>
    </domain-config>
</network-security-config>

Everything else stays HTTPS-only. Traffic to 127.0.0.1 never leaves the phone, so there's nothing to encrypt.

Why a server?¶

The alternative would be a second code path for SMB in every place that shows a file: a Coil fetcher, an ExoPlayer DataSource, a retriever source, each with its own seeking and caching. With an address, SMB files go through exactly the same code as files from any other host. Coil caches them, ExoPlayer streams and seeks them, and the previews and the fullscreen view need no changes.

Why is that safe?

Only this phone can reach it. It's bound to 127.0.0.1, not to the Wi-Fi network.
Only this app knows the token. Another app on the phone could find the port, but without the token every request gets 403. The token is new on every start, so an old URL stops working.
It only reads. GET and HEAD are the only methods; there's no way to change a file through it.