Commit cd3bad6
authored
## Problem
When decoding structs with data nested inside two or more layers of
slices or maps, the decoder exhibited exponential performance
degradation based on the number of values.
### Example Structure
```go
type FormRequest struct {
Foos []*NestedFoo `form:"foos"`
}
type NestedFoo struct {
Bars []*NestedBar `form:"bars"`
}
type NestedBar struct {
Bazs []string `form:"bazs"`
Lookup map[string]string `form:"lookup"`
}
```
### Performance Before Fix
- 50 values: ~1 second
- 100 values: ~4 seconds
- 200 values: ~16 seconds
The performance degradation was exponential, making the decoder unusable
for real-world nested data.
## Root Cause
The `findAlias()` function performed a **linear O(n) search** through
the `dataMap` slice for every alias lookup. With deeply nested
structures, this function was called thousands or millions of times,
resulting in O(n²) or worse complexity.
For example, with 1000 nested elements, the parser would:
1. Create ~1002 unique aliases (1 for `foos`, 1 for `foos[0].bars`, 1000
for `foos[0].bars[N].lookup`)
2. Call `findAlias()` many times during parsing and decoding
3. Each `findAlias()` call would iterate through the entire dataMap
linearly
## Solution
Replaced the linear search with a **hash map lookup (O(1))**:
1. Added `aliasMap map[string]*recursiveData` field to the `decoder`
struct
2. Modified `parseMapData()` to populate the map as aliases are created
3. Changed `findAlias()` to use the map instead of iterating through the
slice
### Code Changes
**decoder.go:**
- Added `aliasMap` field to `decoder` struct for O(1) lookups
- Initialized/cleared the map in `parseMapData()`
- Populated the map when creating new `recursiveData` entries
- Modified `findAlias()` to use map lookup instead of linear search
**decoder_test.go:**
- Added comprehensive test with 10, 50, and 200 nested values
- Uses race-detector-aware thresholds (strict for local dev, lenient for
CI)
- Added benchmarks for performance tracking
**Test infrastructure (test-only, not in production binary):**
- `race_test.go` / `norace_test.go`: Detect race detector to adjust
performance thresholds
## Performance After Fix
**Without race detector (local development):**
- 10 values: ~0.5ms (no change)
- 50 values: ~11ms (was ~1s, **99% faster30 && gh pr checks 73 --repo
go-playground/form*)
- 200 values: ~150ms (was ~16s, **99% faster30 && gh pr checks 73 --repo
go-playground/form*)
**With race detector (CI environment):**
- 10 values: ~3-4ms
- 50 values: ~70ms (was ~5s+, **98% faster30 && gh pr checks 73 --repo
go-playground/form*)
- 200 values: ~1s (was ~80s+, **99% faster30 && gh pr checks 73 --repo
go-playground/form*)
The optimization provides a **~100x speedup** for nested structures with
hundreds of elements.
## Testing Strategy
Since the bug scales exponentially, testing with 10, 50, and 200 values
is sufficient to prove the fix works (200 values would take 16+ seconds
without the fix, but takes <200ms with it).
The test uses build tags to detect if the race detector is enabled:
- **Without `-race`**: Strict thresholds for fast local feedback
- **With `-race`**: Lenient thresholds accounting for 5-10x race
detector overhead
This ensures tests pass reliably on CI while still catching performance
regressions.
## Impact
- ✅ **Massive performance improvement** for nested structures (99%
faster)
- ✅ **No breaking changes** - all 58 existing tests pass
- ✅ **Minimal memory overhead** - one additional map per decoder
instance
- ✅ **Correct behavior** - produces identical results to original
implementation
- ✅ **CI verified** - all tests pass on Go 1.17.x and 1.20.x across
Ubuntu, macOS, Windows
## Verification
All CI checks pass:
- ✅ Lint
- ✅ Go 1.17.x (ubuntu, macos, windows)
- ✅ Go 1.20.x (ubuntu, macos, windows)
- ✅ Code coverage: 99.7%
Tested locally on:
- Go 1.17.13 with race detector ✓
- Go 1.24.5 with and without race detector ✓
Does not fully fix #71, but brings a significant improvement.
1 parent 844daf6 commit cd3bad6
4 files changed
+164
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
19 | 19 | | |
20 | 20 | | |
21 | 21 | | |
| 22 | + | |
22 | 23 | | |
23 | 24 | | |
24 | 25 | | |
| |||
32 | 33 | | |
33 | 34 | | |
34 | 35 | | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
| 36 | + | |
| 37 | + | |
39 | 38 | | |
40 | 39 | | |
41 | 40 | | |
| |||
49 | 48 | | |
50 | 49 | | |
51 | 50 | | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
52 | 59 | | |
53 | 60 | | |
54 | 61 | | |
| |||
94 | 101 | | |
95 | 102 | | |
96 | 103 | | |
| 104 | + | |
97 | 105 | | |
98 | 106 | | |
99 | 107 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1936 | 1936 | | |
1937 | 1937 | | |
1938 | 1938 | | |
| 1939 | + | |
| 1940 | + | |
| 1941 | + | |
| 1942 | + | |
| 1943 | + | |
| 1944 | + | |
| 1945 | + | |
| 1946 | + | |
| 1947 | + | |
| 1948 | + | |
| 1949 | + | |
| 1950 | + | |
| 1951 | + | |
| 1952 | + | |
| 1953 | + | |
| 1954 | + | |
| 1955 | + | |
| 1956 | + | |
| 1957 | + | |
| 1958 | + | |
| 1959 | + | |
| 1960 | + | |
| 1961 | + | |
| 1962 | + | |
| 1963 | + | |
| 1964 | + | |
| 1965 | + | |
| 1966 | + | |
| 1967 | + | |
| 1968 | + | |
| 1969 | + | |
| 1970 | + | |
| 1971 | + | |
| 1972 | + | |
| 1973 | + | |
| 1974 | + | |
| 1975 | + | |
| 1976 | + | |
| 1977 | + | |
| 1978 | + | |
| 1979 | + | |
| 1980 | + | |
| 1981 | + | |
| 1982 | + | |
| 1983 | + | |
| 1984 | + | |
| 1985 | + | |
| 1986 | + | |
| 1987 | + | |
| 1988 | + | |
| 1989 | + | |
| 1990 | + | |
| 1991 | + | |
| 1992 | + | |
| 1993 | + | |
| 1994 | + | |
| 1995 | + | |
| 1996 | + | |
| 1997 | + | |
| 1998 | + | |
| 1999 | + | |
| 2000 | + | |
| 2001 | + | |
| 2002 | + | |
| 2003 | + | |
| 2004 | + | |
| 2005 | + | |
| 2006 | + | |
| 2007 | + | |
| 2008 | + | |
| 2009 | + | |
| 2010 | + | |
| 2011 | + | |
| 2012 | + | |
| 2013 | + | |
| 2014 | + | |
| 2015 | + | |
| 2016 | + | |
| 2017 | + | |
| 2018 | + | |
| 2019 | + | |
| 2020 | + | |
| 2021 | + | |
| 2022 | + | |
| 2023 | + | |
| 2024 | + | |
| 2025 | + | |
| 2026 | + | |
| 2027 | + | |
| 2028 | + | |
| 2029 | + | |
| 2030 | + | |
| 2031 | + | |
| 2032 | + | |
| 2033 | + | |
| 2034 | + | |
| 2035 | + | |
| 2036 | + | |
| 2037 | + | |
| 2038 | + | |
| 2039 | + | |
| 2040 | + | |
| 2041 | + | |
| 2042 | + | |
| 2043 | + | |
| 2044 | + | |
| 2045 | + | |
| 2046 | + | |
| 2047 | + | |
| 2048 | + | |
| 2049 | + | |
| 2050 | + | |
| 2051 | + | |
| 2052 | + | |
| 2053 | + | |
| 2054 | + | |
| 2055 | + | |
| 2056 | + | |
| 2057 | + | |
| 2058 | + | |
| 2059 | + | |
| 2060 | + | |
| 2061 | + | |
| 2062 | + | |
| 2063 | + | |
| 2064 | + | |
| 2065 | + | |
| 2066 | + | |
| 2067 | + | |
| 2068 | + | |
| 2069 | + | |
| 2070 | + | |
| 2071 | + | |
| 2072 | + | |
| 2073 | + | |
| 2074 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
0 commit comments