Skip to content

Conversation

SumitkCodes
Copy link

@SumitkCodes SumitkCodes commented Aug 29, 2025

Description

Fixes the bug reported in #62205 where record_prefix was completely ignored when record_path was empty.

The Problem

When calling pd.json_normalize(data, record_prefix="T") with empty record_path, the record_prefix was ignored due to a performance optimization that bypassed the prefix logic.

Before (broken):

import pandas as pd
_s = pd.Series([{"k": f"{i}", "m": "q"} for i in range(5)])
result = pd.json_normalize(_s, record_prefix="T")
# Result: columns were ["k", "m"] (no prefix applied)

Expected behavior:

# Result: columns should be ["T.k", "T.m"] (prefix correctly applied)

The Solution

Modified the early return optimization logic to exclude cases where record_prefix is provided. When record_prefix is provided, the function now properly applies the prefix using the existing nested_to_record function with correct separator handling.

Code Changes

File: pandas/io/json/_normalize.py

The fix ensures that when record_prefix is provided, the function skips the early return optimization and properly applies the prefix, even when record_path is None.

Testing

Added test case test_record_prefix_without_record_path that verifies the bug is fixed. All existing tests continue to pass.

Verification

import pandas as pd
_s = pd.Series([{"k": f"{i}", "m": "q"} for i in range(5)])
result = pd.json_normalize(_s, record_prefix="T")
print(result.columns.tolist())
# Output: ['T.k', 'T.m']

Checklist

  • I have checked that this issue has not already been reported
  • I have confirmed this bug exists on the latest version of pandas
  • I have provided a minimal, reproducible example
  • I have confirmed this bug exists on the main branch of pandas
  • I have added tests that demonstrate the fix
  • All tests pass

Related Issues

Closes #62205

- Modified early return optimization to exclude cases where record_prefix is provided
- Added proper prefix application using nested_to_record for consistent separator handling
- Added comprehensive test case to verify the fix
- Maintains backward compatibility and performance optimization for cases without record_prefix

Fixes pandas-dev#62205
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: record_prefix ignored when record_path is empty
1 participant