Some Localization Requests (Translateable Jump Nodes and Wing names, Fix Script Implementation remaining)

[NovachenFS2]

Because Mantis is not used these days anymore as it seems, i think, that this is the place to post some Requests for some localization features i need or atleast want to discuss.

As you may know i am constantly working on german translations of Freespace Mods.
For a new generation of updates, i added some new translation features where i also encountered some problems so far.

1. • Regardless that briefing labels are supporting XSTRs now, some entries are automatically translated by the code.
     In the lcl_translate_brief_icon_name_gr() segment (as well for the other languages) of the localize.cpp there are some examples which briefing icons are translated automatically, if the corresponding string occurs.
     Fortunately the game only refers to the strings, that are appearing in the game, so to speak it is using only the already translated strings as a reference (if available). In the german language i actually have only problems with every briefing icon, that is labeled "Transporter" because it is nearly the same word as the english "Transport" what have to be translated by the rules of the code. If i use "Transporter" in tstrings as a translation, the game actually make it a "Transporterer" in the briefing with an additional -ER, what is totally wrong anyway.
     This would not be not a big issue, if i remember to simply translate all "Transport/s" into "Transport" to let the game add the missing -ER by itself. The problem i have is, that this function drops all other characters besides the "Transport" part. If i have a label called "Shivan Transport" for example and translate it to "Shivanischer Transport/er" only the "Transporter/er" will appear. It is dropping the complete part before or after that term. I can only bypass this, if i use a lowercase t at the beginning. But i think, it would be much better to change the code, that these translation rules comes only in effect if the term is totally identical. So that the game still auto translates labels that are called "Transport/s"... but not if they are called "Shivan Transport/s" for example... or already translated terms like "Transporter".
2. • Because i am already in the localize.cpp. Due to the lcl_translate_ship_name_gr() and lcl_translate_targetbox_name_gr() some ship names are translated automatically without breaking the mission. For the creation of a mission that is supporting several languages at once, it would be helpful if there would be a feature to create an alternative name for some of the objects where the original name is used as reference. The ships and weapons.tbl uses the $Alt Name string for that, which make it possible to create translateable weapons (that Alt Name is used in the weapons gauge for example) and ship classes.
     Unfortunately $Alt Name in missions is used for an alternative name for ship classes. But actually because the original game do this with Instructors and Support ships i am asking if it would not be helpful if you can use a new entry to translate objects or wings if it is needed. Well, at least there would be even a potential for new SEXPs where you can change the ship name without any scripts.
3. • I already mentioned the $Alt Name feature in the weapons.tbl. For FreeSpace: The Great War Volition created a hack in the hudtarget.cpp to shorten the name of the Cluster Bomb to Cluster. I think the purpose of the hack was to create a name that is usable in 640x480... but well, because they forget to shorten the name of the Leech Cannon aswell, i do not think, that this hack is worth to be present today. The reason why i am requesting this is, that this Hack prevents that you can use the $Alt Name on this weapon in FSPort. So the name that appears in the weapon gauge is always Cluster and because the $Alt Name refers to a object that has the name "Cluster Bomb" it does not work, also any translated names can not appear aswell.
4. • Actually 3a). Maybe this is a bug or atleast it is unpleasant, but if you target a bomb that have an $Alt Name, the untranslated $Name is displayed instead. So you will always see the english name and not the translated one.
5. • In Missions you can change the $command sender to something of your likings. This feature has the advantage, that it supports XSTR and is translateable on this way. Unfortunately the #Command source you can select as a message source by default is totally independent from that. Why is that? Actually i never had an idea in which cases Command and Command sender could be different... why should Command be a different person to the person that is sending all the default command messages?
     Actually i would like to see a feature, that this setting is also affecting the message source of #Command messages... that would make it possible to create a source for messages that can be named differently in several different languages without to create missions for each language.
6. • Medal Names - Well, this is a problem that is very similar to the tech-intel-xstr sexp. Currently it is not possible to create translated medal names without to change their name in the specific mission. As far as i know this point is already on the agenda, but i post that as an reminder. Next to a grant-medal-xstr SEXP i actually thing, that the $Alt Name solution in the medals.tbl would also work.

I think that was everything so far. I hope at least some of it is possible to implement, because that would make translations much easier and would also help that you need only one file for more than one languages in the future :)

Greetings,

Novachen

[bryansee]

And what about Malay and Chinese languages?

[chief1983]

And what about them? He's not working on those obviously, just German.

[NovachenFS2]

Well, what is about them? There is a Japanese Translation Patch for FreeSpace 2 thanks to the Unicode support... so i think that you are able to do a Malay and Chinese Translation already.

What i am requesting here, is not specific only for the german language... except point 1 which is only present for specific languages (german, french and polish iirc). In Malay and Chinese you would not even have this problem.

[bryansee]

Yeah. But where's the support for Malay and Chinese anyways?

[asarium]

The Unicode changes should be able to support those languages. This issue has nothing to do with that so please stop asking unrelated questions here.

[Goober5000]

For item number 5, the feature should already do what you want. The "command sender" option specifies what is displayed when the "command persona" messages (e.g. "Incoming jump signature! Hostile configuration!") are sent. So, if you set the "command sender" to XSTR("#Command", 7823457) and you have an appropriate tstrings.tbl, then it should automatically change to #Kommandatur or whatever you have.

However this does not apply to senders which are manually supplied using the send-message ship. So using

( send-message
   "#Command"
   "High"
   "eat a sandwich"
)

will still show the message coming from #Command. You may be able to edit the mission using Notepad++ and manually add an XSTR, but I have not tested this.

[NovachenFS2]

Yes, the $command sender can be translated on this way and this is working (maybe i described that too unclear, but i already use this method). But this is not applied to the #Command as message source and in my experience you will likely break SEXPs if you try to add XSTRs into them, because SEXPs are very strict in this cases. Here this would likely cause into a crash of FRED2 due to a wrong used send-message-SEXP. And if not i think an error message about a missing object in the mission with the name XSTR("#Command", 12345), which would already create problems, because of the quotation marks in the middle of it, are more likely.

[asarium]

(1): Do I understand you correctly that the code already supports XSTRs for the briefing labels? I would guess that that is the case in which case it would be pretty dumb to try translating it again.

The old translation code could be disabled if unicode mode is active. Since most translation mods will probably use that this would be a good way for preserving backwards compatibility.
Another solution could be to use the "Target version" feature for enabling this new behavior if a sufficiently new version is targeted. I have no particular preference for either solution so just tell me what you prefer and I'll implement that.

(2): I don't really understand what you are requesting here. Do you want translatable wing names?

(3): The code could be changed to always use the $Alt Name if there is one and only apply the workaround if there is no alternative name. That should fix your issue without changing the old behavior.

(4): I found the issue and fixed it already.

(6): An alternative name for medals is probably the best way to do this. The actual medal name is used in multiple places including the pilot file code so I would like to avoid using a translatable string for the medal name.

[NovachenFS2]

Do I understand you correctly that the code already supports XSTRs for the briefing labels?

Yes, if i remember correctly, you was the person who implement that, because i requested that in Mantis back then. And currently the game tries to translate it again, because the code seems to look for special terms in the labels. If there is a label which contain the word "Transport", the game translate it automatically into "Transporter" in german, even it was already translated in tstrings, because "Transporter" contain the "Transport" aswell.
Well, because you can translate the game into german also without Unicode mode, i am not sure if this would help me very much to deactivate this behaviour if Unicode is used. The only problem i have here is, that the german word "Transporter" is similar to the english word "Transport". If the german translation of this word would be a different one, this would not be even an issue. But because it does, the game see the "Transporter" (even it is called "NTF-Transporter" for example), the code recognize the "Label contains Transport" and translate it then. So the best solution would be, that the game simply does translate only, if there is not a proper tstring id available (so -1)... or the xstr is still missing at all (like in the retail missions).

I don't really understand what you are requesting here. Do you want translatable wing names?

Well, more importantly would be translateable object names in general. For example, in the first FSPort Mission "Eve of Destruction" you have a NavBuoy with the Name "Distress Signal"... a translateable object would make it possible to call it "Notsignal" in german for example as well. "Instructor", "Sentry Gun" and "Support" are objects that are translated automatically by the game. I simply would like to see a feature that enables the translation of other objects as well.

In the past couple of days i worked with the Callsign feature a bit. Actually you can translate callsigns if you put them into XSTRs and also put the translated term into the $callsign list at the beginning of the mission file. Unfortunately the callsign is not used as the actual name in the escort list for example. Even this would still require a changed mission file for every new language you add into a translation, because you have to add the translated callsign into the callsign list, but well, actually a Callsign that can also behave like a real ship name (appear in the escort list as such etc.) (maybe with a new flag?) would also be very helpful already.

[asarium]

Yes, if i remember correctly, you was the person who implement that, because i requested that in Mantis back then. And currently the game tries to translate it again, because the code seems to look for special terms in the labels. If there is a label which contain the word "Transport" label, the game translate it automatically into "Transporter" in german.

Huh, I don't remember writing that code... Anyway, do you have any preference for how the "old" behavior should be disabled?

I'll have to take a look at how the engine handles callsigns to determine how to add proper translation support for ship names. I though we already had that somehow but apparently not...

[Goober5000]

Regarding callsigns, they are not quite the same things as alternate ship names. They are displayed in the message sender as the callsign itself, but on the target display as the ship name plus the callsign. (However I believe the format of the callsign on the target display can be configured.)

The primary issue, as I see it, is that callsigns are not used either in the comm menu or the escort list. I looked into adding this feature a couple months ago but didn't pick it up, as I was quite busy at the time and it is not a trivial problem.

The callsign feature wasn't designed with translation support in mind, but XSTRs should work with callsigns in the same way as XSTRs work anywhere else.

Regarding #Command, @NovachenFS2, have you tested putting XSTR into the send-message sexp as the message sender? I am pretty sure it should work. You will probably not be able to save it in FRED, but as long as you remember to only edit it in Notepad++, it should be fine. There should not be an issue with quotation marks because XSTR parsing is well defined, and there should not be a problem with translated strings because a message source that starts with # can be any string.

In other words, yes, many sexps will break if you just add XSTR in the middle of them, but this particular case is an exception and it should work just fine.

[NovachenFS2]

@asarium

Huh, I don't remember writing that code

It was this one: #1287

Anyway, do you have any preference for how the "old" behavior should be disabled?

No, not really. The question i only have, it is needed to disable the old behavior at all or only to change it?

That is the current code for the position:

`	} else if ((pos = strstr(name, "Transport")) != NULL) {
		pos += 9;		// strlen of "transport"
		strcpy_s(buf, "Transporter");
		strcat_s(buf, pos);
		strcpy(name, buf);`

Actually.. would it not be sufficent already if you would change that into

 	} else if (!stricmp(name, "Transport")) {
		strcpy(name, "Transporter");

	} else if (!stricmp(name, "Transports")) {
		strcpy(name, `"Transporter");

?

Because the strstr function is the main problem, because the translated term also contain the string. But the string comparsion would only be in effect if the label is called like the original english one.

In the polish language it seems to be the same problem. Even i do not know, how accurate the term Transportowiec for a translation of Transport really is, because google translate gives me a total different term for Transport. But i think that could be used in the same way.

Instead of

	} else if ((pos = strstr(name, "Transport")) != NULL) {
		pos += 9;		// strlen of "transport"
		strcpy_s(buf, "Transportowiec");
		strcat_s(buf, pos);
		strcpy(name, buf);

simply this:

 	} else if (!stricmp(name, "Transport")) {
		strcpy(name, "Transportowiec");

	} else if (!stricmp(name, "Transports")) {
		strcpy(name, `"Transportowiec");

All other cases with the "contain string" codes are present for names in the Target box. But if we are able to get deeper into the ships with an alternative translateable name, i think that this cases can be avoided. (except for the support ships so far, because you can not change their name in a mission anyway)

@Goober5000
Yes i know, that callsigns are designed for something different, as their name suggests. But well, in my experiments this one come to the closest to a ship designation that is translateable. And the thought in my idea was, that the extension of this feature, where a mission designer can set a new flag which make a callsign act as an actual ship name, could be easier to implement than to create a whole new one for this purpose.

And no, actually i did not try to use a XSTR in the send-message SEXP. This was simply too offbeat for me, because i know how every other SEXP is reacting to XSTRs.

But because i will test the Scramble Message thing anyway in the course of the day, so i could use the opportunity to test this one as well.

EDIT:
@asarium

I checked all of FreeSpace2 missions for the abovementioned solution with the string comparsion.
Actually this affects only three missions in the game.
sm2-10.fs2 has a label called Transport 1, which looks more like an error to me, that could be fixed with the next assets MVP. Because there is nothing like a Transport 2.

sm3-04.fs2 uses a couple of "Transport" labels.

X_Third-Watch_X.fs2, which is a SOTY-mission, uses a "Transports" label.

@Goober5000
I tried that with the XSTR... it did not work.

Both

   ( send-message 
      XSTR("#Command", 4000) 
      "High" 
      "Mission Done" 
   )

or

   ( send-message 
      "XSTR("#Command", 4000)" 
      "High" 
      "Mission Done" 
   )

results in the same error message.

Error: Required token = [#Goals] or [$Formula:], found [ )

[asarium]

I want to disable the old translation code because I think that it is bad to hard-code translations in the source code since they can't be changed easily. Your suggestions would probably fix the immediate issue but the fundamental issue of hard-coded translations would still exist.

I think it would be better to introduce a mod table option for disabling this behavior which would leave old missions unchanged but would allow you to translate the missions properly. The hard-coded translations wouldn't be applied anymore though so if you rely on that behavior you will have to duplicate the translations but that's only a one-time operation so it shouldn't be too bad.

For the send-message SEXP, have you tried using a SEXP variable for the name? Since you can set the value of the variable from an XSTR the variable value can be translated and the send-message SEXP should get the right message sender. It's obviously not perfect but it would fix the immediate issue until we can figure out a better solution.

[NovachenFS2]

Well okay, i understand this yes.

But i do not have any preference how you make the old code disableable. As long this is not restricted to Unicode Mode only, because most mods/campaigns are not using TTFs. A special mod-table option is fine, yes.

For the send-message SEXP, have you tried using a SEXP variable for the name? Since you can set the value of the variable from an XSTR the variable value can be translated and the send-message SEXP should get the right message sender. It's obviously not perfect but it would fix the immediate issue until we can figure out a better solution.

Nope, i did not try that, because i actually do not have much fredding experience with Variables and i had already some problems to understand a finished mission with all the Variable Stuff for my first translation (BtA).
But i have to agree, that this could be worth a try.

[Goober5000]

@NovachenFS2 Strange, I thought the XSTR in the middle of the SEXP would work. I was mistaken. Thank you for testing.

I think @asarium's suggestion of using a SEXP variable is the right idea. It is not really possible to modify the send-message SEXP so we would have to add a new send-message-sender-xstr SEXP. And we would also need to modify any other SEXP which had this problem. However, the SEXP string variable solution can be used for any SEXP that exists now or will be created in the future. The only thing we might need to do is add a new modify-variable-xstr SEXP for storing the translation into the variable.

[asarium]

The modify-variable-xstr SEXP already exists since I added it as part of another batch of changes requested by @NovachenFS2. I still think it would be better to also have support for translating SEXP parameter strings without adding new SEXP variables since those count towards the variable limit.

Maybe a special string syntax similar to how SEXP variables could be added which contains the untranslated string and the id somehow? I am not very familiar with the SEXP system so I don't know how feasible that would be.

[NovachenFS2]

I tried it with the variable as a message source and yes, it worked!
That is already a very big help actually to make releases which only need one mission file for several languages. So this is a good solution for 5). The need for different mission files is now only required in cases, where the mission mission have objects, which name should be translated.

In this process i looked a little more in the Variable translation in general, because for more recent mods i have to implement translated variables anyway in the missions. Found a small problem in its behaviour.

If a string is required to end with a space (to seperate the text to other values that directly follows) the space in the translation is ignored. Only the original/untranslated string is working as it should. Two spaces at the end as a workaround make no difference. Also there seems to be no character in Unicode, that can be used as a invisible placeholder like the well known #, And all unsupported characters from the TTF are shown as a square and so can not be used to write an invisible character.

But to summerize all the points of the OP so far, based on the previous posts:

1. • The automatic translation will be deactivateable by a setting in the game_settings.tbl/-mod.tbm.
2. • Still open.
3. • The old Cluster Bomb hack by Volition will be deleted.
4. • Alt Name, if present, will be displayed first.
5. • Can already be reached with translated variables.
6. • There will be $Alt Name in the medals.tbl like it is in ships and weapons.tbl.

[asarium]

For (2): I think it would be a good solution to also introduce a "Display Name" property of a ship which is used for modifying the display name of the ship without actually changing the "name" of the ship. The callsign feature is not meant to be an alternate name of the ship so it would not be a good idea to reuse this field for translating ship names.

I have opened a PR with the changes: #1736 @NovachenFS2 Please test the changes to make sure that they really fix the issues you have reported.

[NovachenFS2]

Yes, the "Display Name" was the solution i would had prefered right from the start. So yes, i am very fine with this. The Callsign was only an idea to create a solution until there is a better one :).

What i have to enter in the mod table to disable the built-in translations?

"$Disable built-in translations: Yes" in #Other Settings?

[asarium]

Yes $Disable built-in translations: needs to be added as the last option in your mod table.

For medals, $Alt Name: needs to be added after $Name:.

[asarium]

@NovachenFS2 Could you open a separate issue for the bugs you have found in the Unicode code maybe with a test mod to reproduce the issue? Also, there is the zero width space in Unicode which should work with FSO if you need to add an invisible character for some reason.

[Goober5000]

If modify-variable-xstr already exists, fantastic. :) We have so many SEXPs now that I'm starting to lose track of some of them.

At a minimum, in order to add support for XSTRs as string parameters we would need to create a new SEXP parameter type, OPF_XSTR which should be compatible with OPF_STRING. But the SEXP node would need some sort of flag to distinguish between an XSTR and a non-XSTR string. Furthermore, the event editor would need some sort of pop-up edit box to set the XSTR index. Fortunately the internal SEXP infrastructure would not need to be changed, as instead of storing the string, we could store the XSTR index.

I heartily agree with adding the Display Name feature for ship names. This could also have some fun applications in FRED outside of the realm of translations.

[asarium]

I implemented most of the display name feature but event screen support is still missing since the same data is used by various SEXPs which use the actual ship names.

[Goober5000]

Meaning the mission log? Yeah, that's going to be tricky.

[asarium]

I think I managed to get the mission log working. I needed to do some changes to the log entry structure but it seems to be working fine now.

[asarium]

@NovachenFS2 Now that the ship display name feature is merged, what other things are needed to fully resolve this?

I know you mentioned translating wing and jump node names. Anything else?

[Goober5000]

Pinging @NovachenFS2 again

[NovachenFS2]

I'm sorry. I did not see the message. @asarium

Translateable Jump Nodes and Wing names is the last feature that is missing from the list i had.

What is missing apart from that... well actually it should be implemented already with #1527, but as far as i tried, i was not able to implement translateable text in script files, even i try to keep the structure that is used in the commit.
So i am not sure if i simply use it wrong (what is possible indeed) or it does not work as it should.
Maybe i simple need a sample to use it properly.

[JohnAFernandez]

I updated the name to make it clearer what is left at a glance for whoever might take it on.

[MjnMixael]

All that's left on this now is translatable wing names.