Skip to content

Localized voice files, fonts and SCPUI files via language subfolders - #8062

Open
Goober5000 wants to merge 6 commits into
scp-fs2open:masterfrom
Goober5000:fix/language_subdir_lookup
Open

Goober5000 wants to merge 6 commits into
scp-fs2open:masterfrom
Goober5000:fix/language_subdir_lookup

Conversation

@Goober5000

@Goober5000 Goober5000 commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

The engine has long had a way to load a localized copy of a file from a subfolder named after the language's extension, e.g. data/fonts/gr/font01.vf for German. Fonts were its only user, and since cfile gained subfolder support (#4771) the scheme has had a hole. This PR fixes the hole, then extends the scheme to voice files, TrueType fonts and SCPUI files.

A translation or voice pack can now ship its files next to the originals, under the same names:

data/voice/briefing/TR1_MB_01.ogg       original
data/voice/briefing/gr/TR1_MB_01.ogg    played when German is the current language

With no localized copy, the original plays. English, retail hybrid and untranslated have no extension, so for them nothing changes and no extra lookups happen.

1. Keep bare-name lookups out of language subfolders

Since #4771, a lookup that doesn't name a subfolder matches a file in any subfolder, and the first match in root order wins. So a higher-priority mod that shipped data/fonts/gr/font01.vf or data/voice/briefing/gr/x.ogg replaced the original for every player, whatever their language. #4771 noted that localization would need changes to the subfolder code; this is that change.

Lookups and listings that don't name a subfolder now skip files whose innermost folder is the extension of any language in Lcl_languages. Naming the subfolder (gr/font01.vf) or using a */ glob still finds them. Ordinary subfolders and override priority are unchanged.

2. Find training voices with any audio extension (retail bug)

message_play_training_voice() opened the file by its exact name to decide whether to stream it or load it. Both loaders accept any audio extension, but the size check didn't, so a mission naming x.wav with only x.ogg on disk played no training voice. The size now comes from an extension-agnostic lookup. This is independent of localization, and it's committed separately.

3. Load voice files from the language subfolder

New cf_localize_filename() returns the name to load: the localized copy if it exists, otherwise the name as given. It's used by:

  • audiostream_open() for ASF_VOICE streams: briefings, command briefings, debriefings, the fiction viewer, red alert, loop briefs, long training voices and Lua AUDIOSTREAM_VOICE streams;
  • message_load_wave(), for mission and built-in messages;
  • the short training voice, and the training size check, so the stream-or-load decision is made on the file that will play;
  • the debriefing persona check and the FS1-to-Command check, so a variant that exists only in the language subfolder counts.

Design decisions:

  • The text language picks the voice language. There's no separate setting.
  • The localized copy wins wherever it's found. Overrides go by filename, as for any other file. A mod that replaces x.ogg replaces only x.ogg; it doesn't hide the base game's gr/x.ogg. A mod that wants to replace the localized line ships its own gr/x.ogg. This matches how fonts already behaved, and how text works: a mod's new lines aren't translated until it supplies translations.
  • Names stay within MAX_FILENAME_LEN. If the localized name would be 32 characters or longer, the original is used and a line is logged.

4. Bring back cfopen()'s localize flag; TrueType fonts

_cfopen() has carried an unused bool localize parameter since #4771. When reading, it now opens the localized copy if there is one and it satisfies the caller's location flags, and otherwise the file as named. Write modes ignore it. Every existing caller passes false; they only fill that position to reach location_flags, and they all open user or generated files (pilot and campaign files, control presets, the shader cache).

Both font loaders now use cfopen(name, "rb", CF_TYPE_ANY, true). .vf fonts behave as before, but the hand-written fallback is gone. .ttf and .otf fonts are localized for the first time, so a translation can ship a font with the glyphs it needs (data/fonts/tw/<name>.ttf) without changing anyone else's fonts.

5. SCPUI files

RocketFileInterface::Open() now opens every file with localize, so a translation can ship its own RML documents, stylesheets and fonts in the language subfolder. SCPUI paths only choose the path type, and cfile finds the file by name, so the same rule applies as for other localized files. Images are loaded through bmpman and aren't affected.

load_fonts() also lists the current language's font subfolder, so a font that exists only there is still loaded. libRocket picks fonts by the family name inside the file, so a localized font must keep the original's family name to replace it.

Notes for mod authors

  • A language's subfolder name is its +Extension: in strings.tbl or a *-lcl.tbm. The built-in German, French and Polish use gr, fr and pl.
  • Folders named like a language extension are now reserved. A mod that kept unrelated files in a folder named gr, fr, pl or one of its own language extensions can no longer reach them by bare name.
  • The localized name, including the subfolder, must be under 32 characters. With a two-letter extension, that leaves 28 for the filename.
  • The language folder goes directly inside the type's folder: data/voice/briefing/gr/x.ogg. If the original is in an ordinary subfolder (data/voice/briefing/act1/x.ogg), the localized copy still goes in data/voice/briefing/gr/, not act1/gr/. Only the innermost folder counts as a language folder, so files in gr/act1/ are ordinary files that every player gets.
  • For a localized message wave, the Lua Filename of the message's voice file includes the subfolder (gr\x.wav).

🤖 Generated with Claude Code

@Goober5000 Goober5000 added this to the Release 27.0 milestone Nov 17, 2026
@Goober5000 Goober5000 added enhancement A new feature or upgrade of an existing feature to add additional functionality. fix A fix for bugs, not-a-bugs, and/or regressions. localization / translation (XSTR) A feature or issue related to language localization and translations. labels Nov 17, 2026
@Goober5000

Copy link
Copy Markdown
Contributor Author

@caasi @NovachenFS2 @Achille44 this should be relevant to your interests

@caasi

caasi commented Nov 17, 2026

Copy link
Copy Markdown

Thanks, this covers most of #8054. The part it leaves out is a voice language separate from the text language (for example, English text with a Chinese voice). Would you be OK with me adding that as a small follow-up PR on top of this one?

@Goober5000

Copy link
Copy Markdown
Contributor Author

Thanks, this covers most of #8054. The part it leaves out is a voice language separate from the text language (for example, English text with a Chinese voice). Would you be OK with me adding that as a small follow-up PR on top of this one?

I'd be OK with that. I am a little surprised that anyone would want to do that, since it sounds like a recipe for being very confused. But it can be an option.

@Goober5000
Goober5000 force-pushed the fix/language_subdir_lookup branch from 8c63aeb to f0acaaa Compare November 17, 2026 07:28
@caasi

caasi commented Nov 17, 2026

Copy link
Copy Markdown

I am a little surprised that anyone would want to do that, since it sounds like a recipe for being very confused. But it can be an option.

I'd guess it's because voice acting is not that strong here in Taiwan. China used to be like Taiwan, but voice acting is a real industry there now, with many good young voice actors.

In my experience, games like the MGS or The Division series have no Chinese voice option, so I pick the English or Japanese voices and read Chinese text, to enjoy the work of professional voice actors.

I'm excited to see this PR merged. I will reorganize my mod to follow your folder structure.

@NovachenFS2

NovachenFS2 commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Thanks, this covers most of #8054. The part it leaves out is a voice language separate from the text language (for example, English text with a Chinese voice). Would you be OK with me adding that as a small follow-up PR on top of this one?

I'd be OK with that. I am a little surprised that anyone would want to do that, since it sounds like a recipe for being very confused. But it can be an option.

I actually support this, it is very common in many countries, to have localized Text to english Voice acting.

Also in FS2 the official German localization only covers Voice Files for Command Briefing and In-Mission.
So no voiced Briefing and Debriefing whatssoever.
So people should be able to decide If they want partial german voices, with the rest of the voice in english or a complete english voice acting.

@dustinspace217 dustinspace217 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at f0acaaa. The fix and the voice lookup both work in the game for the layout the description shows. Two layouts behave differently from what I expected, and one font path is left out. None of it blocks; they're yours to weigh.

Tested (Debug builds of master 5af50bd and this head, retail FS2 data plus a small test mod, Linux, OpenGL, sound on). The mod's originals are 1-second tones and its gr copies are 3-second tones, and I asked the engine through Lua whether a bare name exists and how long the voice stream it opens is. German was set with -language German; the mod carries a first font with an unrecognized checksum, because auto-detection otherwise wins over that flag.

  • voice/briefing/x1.ogg plus voice/briefing/gr/x1.ogg: on master an English player gets the German copy (the hole). With this PR English gets the original and German gets the copy.
  • voice/briefing/gr/x3.ogg with no original: master finds it for everyone. With this PR English doesn't find it and German plays it.
  • tables/gr/zzlang-test.tbm: found by bare name on master, not with this PR, as the mod author note says.
  • The four CFileLanguageSubdirTest tests pass, with the rest of the cfile suites.

Two layouts that surprised me (details inline):

  • An original in a subfolder, voice/briefing/act1/x2.ogg with act1/gr/x2.ogg beside it: German still gets the original, on master and with this PR.
  • A language folder that isn't the innermost one, voice/special/gr/act1/x4.ogg: still found by bare name in English with this PR.

From reading, not run:

  • SCPUI loads its fonts by listing *.ttf and *.otf in load_fonts() (rocket_ui.cpp:137) and opens them through RocketFileInterface with a plain cfopen. A font that exists only in data/fonts/tw/ is no longer listed, so SCPUI never loads it, and a same-named copy there is never swapped in. So the data/fonts/tw/<name>.ttf note holds for fonts.tbl fonts but not for SCPUI.
  • cf_get_file_list_preallocated() has no sub-path filter, so pilot and squad images in a language folder are listed but can't be loaded by bare name.
  • Lua message.VoiceFile.Filename now returns gr/x.wav for a localized message voice.

Not tested: fonts, mission messages and training voices in a mission, French and Polish, a mod-defined language, files inside a .vp, Windows.

// except that localized files are only found by naming their language subfolder
if (search.empty()) {
return true;
return index.empty() || !is_language_sub_path(index);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the innermost folder is checked, and the localized lookup asks for exactly gr/<name>. Two results from my test mod:

  • voice/briefing/act1/x2.ogg with act1/gr/x2.ogg next to it: the copy is hidden from the bare lookup here, and cf_localize_filename() looks for gr/x2.ogg, which doesn't match the sub path act1/gr/. German plays the original and nothing is logged. So "next to the originals" works at the top of a folder like voice/briefing, but not for originals a mod keeps in subfolders.
  • voice/special/gr/act1/x4.ogg: the innermost folder is act1, so this still matches a bare x4.ogg for every player.

If both are intended, a line in the mod author notes would cover it.

Goober5000 and others added 4 commits November 17, 2026 21:17
A lookup that doesn't name a subfolder matches a file in any subfolder,
first match in root order.  So a mod that ships localized files in a
language subfolder (data/fonts/gr/font01.vf, data/voice/briefing/tw/...)
replaced the originals for every player, whatever language they chose.
lcl_add_dir_to_path_with_filename() and its font fallback depend on the
bare lookup finding only the original.

The subfolder support from scp-fs2open#4771 noted that file localization conflicts
with it and would need changes to the subfolder code; this is that
change.  Bare lookups and listings now skip files whose innermost folder
is named after the extension of any language in Lcl_languages.  Naming
the subfolder (gr/font01.vf) or a */ glob still finds them.  Ordinary
subfolders and override priority are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
message_play_training_voice() opened the file by its exact name to
decide whether to stream it or load it.  The loaders it chooses between
accept any audio extension, but the size check didn't, so a mission that
named x.wav with only x.ogg on disk played no training voice.  Look the
file up with the audio extension list instead, and take the size from
the lookup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With a language that has an extension, voice files now load from the
language subfolder when a copy exists there, like fonts: German players
get data/voice/briefing/gr/x.ogg in place of data/voice/briefing/x.ogg.
Without a localized copy, the original plays.

The localized copy wins wherever it is found.  Overrides go by file
name, so a mod that replaces x.ogg doesn't hide the base game's
gr/x.ogg; a mod that wants to replace the localized line ships its own
gr/x.ogg.

cf_localize_filename() returns the name to load.  It is used by voice
streams in audiostream_open() (briefings, command briefings,
debriefings, the fiction viewer, red alert, loop briefs, long training
voices and Lua voice streams), by message_load_wave(), by the short
training voice, and by the debriefing persona and FS1 Command checks,
so a file that exists only in the language subfolder counts.  The text
language selects the voice language.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Only .vf fonts were localized: loadFontOld() tried gr/<name> by hand
before the bare name, while loadNVGFont() opened .ttf and .otf files as
named.  Both loaders now open their files with cfopen()'s localize flag,
so a translation can ship a font with the glyphs it needs
(data/fonts/tw/<name>.ttf) without changing the fonts other players get.

The localize parameter of cfopen() was left unused when cfile gained
subfolder support (scp-fs2open#4771).  It now opens the copy given by
cf_localize_filename() when reading, if that copy satisfies the
location flags, and otherwise the file as named.  Every existing caller
passes false.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Goober5000
Goober5000 force-pushed the fix/language_subdir_lookup branch from f0acaaa to 4e9a1dc Compare November 18, 2026 02:17
@Goober5000 Goober5000 changed the title Localized voice files and fonts via language subfolders Localized voice files, fonts and SCPUI files via language subfolders Nov 18, 2026
@Goober5000

Copy link
Copy Markdown
Contributor Author

@dustinspace217 Thanks for the thorough testing.

  • Original in a subfolder (act1/x2.ogg with act1/gr/x2.ogg): intended for now. The localized copy goes in the type's own language folder (voice/briefing/gr/x2.ogg), even when the original is in a subfolder. Supporting act1/gr/ would need cfile to report which subfolder it found the original in. That's doable, but nobody has needed it, so I've left it for later. The layout is documented in the mod-author notes and in cf_localize_filename()'s comment.
  • gr/act1/x4.ogg: also intended. The language folder has to come directly before the filename, so gr/act1/ is an ordinary folder. That's documented too.
  • SCPUI fonts: fixed, and broadened. RocketFileInterface::Open() now opens every SCPUI file with localize, so documents, stylesheets and fonts can all be localized. load_fonts() also lists the current language's font folder, so a font that exists only there is loaded. Images are loaded through bmpman, so they aren't covered. libRocket picks fonts by family name, so a localized SCPUI font has to keep the original's family name.
  • cf_get_file_list_preallocated(): fixed. Its index loop now skips language folders, like the other listings, and there's a test for it.
  • Lua Filename: intended. It's the name the sound was loaded under, and it's noted in the description.

@dustinspace217 dustinspace217 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at 4e9a1dc. The fixes and the doc notes look right to me, but this head doesn't build with gcc or clang. The Linux and Mac CI jobs all fail on it; the Windows ones pass.

rocket_ui.cpp:155 calls SCP_vector_contains_lcase() with a string, and that template in vmallocator.h:65 calls lcase_equal(), which is only declared further down the header (line 118). MSVC accepts that; gcc and clang don't. A declaration above the template fixes it. I tested with extern bool lcase_equal(const std::string& _Left, const std::string& _Right); placed just before it.

With that one line added (Debug, Linux, OpenGL, retail data plus the same test mod as last time), English and German give the same results as in my first review, and the 46 cfile and language unit tests pass, including the new checks in language_subdir_only_localized. I didn't test the SCPUI changes in game.

@Goober5000
Goober5000 force-pushed the fix/language_subdir_lookup branch from 4e9a1dc to bf0cc8b Compare November 18, 2026 05:13
@Goober5000

Copy link
Copy Markdown
Contributor Author

@dustinspace217 fixed compilation

@dustinspace217 dustinspace217 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at bf0cc8b. The compile fix works: gcc builds this head without warnings, and the 46 cfile and language unit tests pass.

One thing left for CI: clang-tidy flags list_fonts in rocket_ui.cpp as a static definition inside an anonymous namespace. That's the only failing job; the other Linux jobs were cancelled after it.

Comment thread code/scpui/rocket_ui.cpp Outdated

void load_fonts()
// the fonts with this extension, including those that only exist in the current language's subfolder
static SCP_vector<SCP_string> list_fonts(const char* ext)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy (readability-static-definition-in-anonymous-namespace): the anonymous namespace already makes this internal.

Suggested change
static SCP_vector<SCP_string> list_fonts(const char* ext)
SCP_vector<SCP_string> list_fonts(const char* ext)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed again

RocketFileInterface::Open() now opens files with cfopen()'s localize
flag, so a translation can ship its own copy of an RML document, a
stylesheet or a font in the language subfolder.  SCPUI paths only choose
the path type and cfile finds the file by name, so this follows the same
rule as other localized files.  Images are loaded through bmpman, so
they are not affected.

load_fonts() also lists the fonts in the current language's subfolder,
because the bare listing skips them, and a font that exists only there
would otherwise not be loaded.  libRocket picks fonts by the family name
stored in the file, so a localized font has to keep the original's
family name to replace it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Goober5000
Goober5000 force-pushed the fix/language_subdir_lookup branch from bf0cc8b to e60ade5 Compare November 18, 2026 07:05

@dustinspace217 dustinspace217 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-checked at e60ade5. The only change since bf0cc8b is the static removal on list_fonts, which is what clang-tidy wanted. Nothing else from my side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement A new feature or upgrade of an existing feature to add additional functionality. fix A fix for bugs, not-a-bugs, and/or regressions. localization / translation (XSTR) A feature or issue related to language localization and translations.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants