Skip to content

Commit b420fe6

Browse files
committed
feat: Add MaxTokens option and i18n support for language variants
- Add MaxTokens option for AI model output control - Add i18n support for language variants (pt-BR/pt-PT)
1 parent 96481f7 commit b420fe6

File tree

19 files changed

+575
-20
lines changed

19 files changed

+575
-20
lines changed

.vscode/settings.json

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
"Callirhoe",
1919
"Callirrhoe",
2020
"Cerebras",
21+
"colour",
2122
"compadd",
2223
"compdef",
2324
"compinit",
@@ -129,6 +130,7 @@
129130
"opencode",
130131
"opencontainers",
131132
"openrouter",
133+
"organise",
132134
"Orus",
133135
"osascript",
134136
"otiai",

CHANGELOG.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
# Changelog
22

3+
## v1.4.317 (2025-09-21)
4+
5+
### PR [#1778](https://github.com/danielmiessler/Fabric/pull/1778) by [ksylvan](https://github.com/ksylvan): Add Portuguese Language Variants Support (pt-BR and pt-PT)
6+
7+
- Add Brazilian Portuguese (pt-BR) translation file
8+
- Add European Portuguese (pt-PT) translation file
9+
- Implement BCP 47 locale normalization system
10+
- Create fallback chain for language variants
11+
- Add default variant mapping for Portuguese
12+
313
## v1.4.316 (2025-09-20)
414

515
### PR [#1777](https://github.com/danielmiessler/Fabric/pull/1777) by [ksylvan](https://github.com/ksylvan): chore: remove garble installation from release workflow

cmd/fabric/version.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,3 @@
11
package main
22

3-
var version = "v1.4.316"
3+
var version = "v1.4.317"
0 Bytes
Binary file not shown.

docs/i18n-variants.md

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Language Variants Support in Fabric
2+
3+
## Current Implementation
4+
5+
As of this update, Fabric supports Portuguese language variants:
6+
7+
- `pt-BR` - Brazilian Portuguese
8+
- `pt-PT` - European Portuguese
9+
- `pt` - defaults to `pt-BR` for backward compatibility
10+
11+
## Architecture
12+
13+
The i18n system supports language variants through:
14+
15+
1. **BCP 47 Format**: All locales are normalized to BCP 47 format (language-REGION)
16+
2. **Fallback Chain**: Regional variants fall back to base language, then to configured defaults
17+
3. **Default Variant Mapping**: Languages without base files can specify default regional variants
18+
4. **Flexible Input**: Accepts both underscore (pt_BR) and hyphen (pt-BR) formats
19+
20+
## Recommended Future Variants
21+
22+
Based on user demographics and linguistic differences, these variants would provide the most value:
23+
24+
### High Priority
25+
26+
1. **Chinese Variants**
27+
- `zh-CN` - Simplified Chinese (Mainland China)
28+
- `zh-TW` - Traditional Chinese (Taiwan)
29+
- `zh-HK` - Traditional Chinese (Hong Kong)
30+
- Default: `zh``zh-CN`
31+
- Rationale: Significant script and vocabulary differences
32+
33+
2. **Spanish Variants**
34+
- `es-ES` - European Spanish (Spain)
35+
- `es-MX` - Mexican Spanish
36+
- `es-AR` - Argentinian Spanish
37+
- Default: `es``es-ES`
38+
- Rationale: Notable vocabulary and conjugation differences
39+
40+
3. **English Variants**
41+
- `en-US` - American English
42+
- `en-GB` - British English
43+
- `en-AU` - Australian English
44+
- Default: `en``en-US`
45+
- Rationale: Spelling differences (color/colour, organize/organise)
46+
47+
4. **French Variants**
48+
- `fr-FR` - France French
49+
- `fr-CA` - Canadian French
50+
- Default: `fr``fr-FR`
51+
- Rationale: Some vocabulary and expression differences
52+
53+
5. **Arabic Variants**
54+
- `ar-SA` - Saudi Arabic (Modern Standard)
55+
- `ar-EG` - Egyptian Arabic
56+
- Default: `ar``ar-SA`
57+
- Rationale: Significant dialectal differences
58+
59+
6. **German Variants**
60+
- `de-DE` - Germany German
61+
- `de-AT` - Austrian German
62+
- `de-CH` - Swiss German
63+
- Default: `de``de-DE`
64+
- Rationale: Minor differences, mostly vocabulary
65+
66+
## Implementation Guidelines
67+
68+
When adding new language variants:
69+
70+
1. **Determine the Base**: Decide which variant should be the default
71+
2. **Create Variant Files**: Copy base file and adjust for regional differences
72+
3. **Update Default Map**: Add to `defaultLanguageVariants` if needed
73+
4. **Focus on Key Differences**:
74+
- Technical terminology
75+
- Common UI terms (file/ficheiro, save/guardar)
76+
- Date/time formats
77+
- Currency references
78+
- Formal/informal address conventions
79+
80+
5. **Test Thoroughly**: Ensure fallback chain works correctly
81+
82+
## Adding a New Variant
83+
84+
To add a new language variant:
85+
86+
1. Copy the base language file:
87+
88+
```bash
89+
cp locales/es.json locales/es-MX.json
90+
```
91+
92+
2. Adjust translations for regional differences
93+
94+
3. If this is the first variant for a language, update `i18n.go`:
95+
96+
```go
97+
var defaultLanguageVariants = map[string]string{
98+
"pt": "pt-BR",
99+
"es": "es-MX", // Add if Mexican Spanish should be default
100+
}
101+
```
102+
103+
4. Add tests for the new variant
104+
105+
5. Update documentation
106+
107+
## Language Variant Naming Convention
108+
109+
Follow BCP 47 standards:
110+
111+
- Language code: lowercase (pt, es, en)
112+
- Region code: uppercase (BR, PT, US)
113+
- Separator: hyphen (pt-BR, not pt_BR)
114+
115+
Input normalization handles various formats, but files and internal references should use BCP 47.
116+
117+
## Testing Variants
118+
119+
Test each variant with:
120+
121+
```bash
122+
# Direct specification
123+
fabric --help -g=pt-BR
124+
fabric --help -g=pt-PT
125+
126+
# Environment variable
127+
LANG=pt_BR.UTF-8 fabric --help
128+
129+
# Fallback behavior
130+
fabric --help -g=pt # Should use pt-BR
131+
```
132+
133+
## Maintenance Considerations
134+
135+
When updating translations:
136+
137+
1. Update all variants of a language together
138+
2. Ensure key parity across all variants
139+
3. Test fallback behavior after changes
140+
4. Consider using translation memory tools for consistency

internal/i18n/i18n.go

Lines changed: 89 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,22 @@ var (
2525
initOnce sync.Once
2626
)
2727

28+
// defaultLanguageVariants maps language codes without regions to their default regional variants.
29+
// This is used when a language without a base file is requested.
30+
var defaultLanguageVariants = map[string]string{
31+
"pt": "pt-BR", // Portuguese defaults to Brazilian Portuguese for backward compatibility
32+
// Note: We currently have base files for these languages, but if we add regional variants
33+
// in the future, these defaults will be used:
34+
// "de": "de-DE", // German would default to Germany German
35+
// "en": "en-US", // English would default to US English
36+
// "es": "es-ES", // Spanish would default to Spain Spanish
37+
// "fa": "fa-IR", // Persian would default to Iran Persian
38+
// "fr": "fr-FR", // French would default to France French
39+
// "it": "it-IT", // Italian would default to Italy Italian
40+
// "ja": "ja-JP", // Japanese would default to Japan Japanese
41+
// "zh": "zh-CN", // Chinese would default to Simplified Chinese
42+
}
43+
2844
// Init initializes the i18n bundle and localizer. It loads the specified locale
2945
// and falls back to English if loading fails.
3046
// Translation files are searched in the user config directory and downloaded
@@ -35,26 +51,30 @@ var (
3551
func Init(locale string) (*i18n.Localizer, error) {
3652
// Use preferred locale detection if no explicit locale provided
3753
locale = getPreferredLocale(locale)
54+
// Normalize the locale to BCP 47 format (with hyphens)
55+
locale = normalizeToBCP47(locale)
3856
if locale == "" {
3957
locale = "en"
4058
}
4159

4260
bundle := i18n.NewBundle(language.English)
4361
bundle.RegisterUnmarshalFunc("json", json.Unmarshal)
4462

45-
// load embedded translations for the requested locale if available
63+
// Build a list of locale candidates to try
64+
locales := getLocaleCandidates(locale)
65+
66+
// Try to load embedded translations for each candidate
4667
embedded := false
47-
if data, err := localeFS.ReadFile("locales/" + locale + ".json"); err == nil {
48-
_, _ = bundle.ParseMessageFileBytes(data, locale+".json")
49-
embedded = true
50-
} else if strings.Contains(locale, "-") {
51-
// Try base language if regional variant not found (e.g., es-ES -> es)
52-
baseLang := strings.Split(locale, "-")[0]
53-
if data, err := localeFS.ReadFile("locales/" + baseLang + ".json"); err == nil {
54-
_, _ = bundle.ParseMessageFileBytes(data, baseLang+".json")
68+
for _, candidate := range locales {
69+
if data, err := localeFS.ReadFile("locales/" + candidate + ".json"); err == nil {
70+
_, _ = bundle.ParseMessageFileBytes(data, candidate+".json")
5571
embedded = true
72+
locale = candidate // Update locale to what was actually loaded
73+
break
5674
}
5775
}
76+
77+
// Fall back to English if nothing was loaded
5878
if !embedded {
5979
if data, err := localeFS.ReadFile("locales/en.json"); err == nil {
6080
_, _ = bundle.ParseMessageFileBytes(data, "en.json")
@@ -158,3 +178,63 @@ func tryGetMessage(locale, messageID string) string {
158178
}
159179
return ""
160180
}
181+
182+
// normalizeToBCP47 normalizes a locale string to BCP 47 format.
183+
// Converts underscores to hyphens and ensures proper casing (language-REGION).
184+
func normalizeToBCP47(locale string) string {
185+
if locale == "" {
186+
return ""
187+
}
188+
189+
// Replace underscores with hyphens
190+
locale = strings.ReplaceAll(locale, "_", "-")
191+
192+
// Split into parts
193+
parts := strings.Split(locale, "-")
194+
if len(parts) == 1 {
195+
// Language only, lowercase it
196+
return strings.ToLower(parts[0])
197+
} else if len(parts) >= 2 {
198+
// Language and region (and possibly more)
199+
// Lowercase language, uppercase region
200+
parts[0] = strings.ToLower(parts[0])
201+
parts[1] = strings.ToUpper(parts[1])
202+
return strings.Join(parts[:2], "-") // Return only language-REGION
203+
}
204+
205+
return locale
206+
}
207+
208+
// getLocaleCandidates returns a list of locale candidates to try, in order of preference.
209+
// For example, for "pt-PT" it returns ["pt-PT", "pt", "pt-BR"] (where pt-BR is the default for pt).
210+
func getLocaleCandidates(locale string) []string {
211+
candidates := []string{}
212+
213+
if locale == "" {
214+
return candidates
215+
}
216+
217+
// First candidate is always the requested locale
218+
candidates = append(candidates, locale)
219+
220+
// If it's a regional variant, add the base language as a candidate
221+
if strings.Contains(locale, "-") {
222+
baseLang := strings.Split(locale, "-")[0]
223+
candidates = append(candidates, baseLang)
224+
225+
// Also check if the base language has a default variant
226+
if defaultVariant, exists := defaultLanguageVariants[baseLang]; exists {
227+
// Only add if it's different from what we already have
228+
if defaultVariant != locale {
229+
candidates = append(candidates, defaultVariant)
230+
}
231+
}
232+
} else {
233+
// If this is a base language without a region, check for default variant
234+
if defaultVariant, exists := defaultLanguageVariants[locale]; exists {
235+
candidates = append(candidates, defaultVariant)
236+
}
237+
}
238+
239+
return candidates
240+
}

0 commit comments

Comments
 (0)