fix(csv): open input files as utf-8-sig to strip BOM transparently

[gkorland]

Summary

The CSV input files were opened without an explicit encoding (io.open(filename, "rt") / open(filename, "rt")), so on Windows the system ANSI codepage was used and UTF-8-with-BOM files (the default for Excel "Save as CSV (UTF-8)") had \ufeff prepended to the first column name. This silently broke:

• schema-mode header parsing (the first name:TYPE field was now \ufeffname:TYPE),
• the "underscore-prefix means non-property ID" heuristic in Label.process_schemaless_header,
• the column-name → property mapping for the affected first column generally.

Changes

Use encoding="utf-8-sig" everywhere a CSV is opened. utf-8-sig transparently strips a leading BOM if present and behaves like plain utf-8 otherwise.

• falkordb_bulk_loader/entity_file.py: EntityFile.__init__ opens self.infile with encoding="utf-8-sig".
• falkordb_bulk_loader/bulk_update.py: same for count_entities and process_update_csv.

Testing

Lint clean (flake8, black).

Memory / Performance Impact

Negligible; utf-8-sig only adds a single optional 3-byte BOM check at file open.

Related Issues

From the comprehensive code-review report (BUG-6).

[coderabbitai]

Warning

Rate limit exceeded

@gkorland has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 47 minutes and 53 seconds before requesting another review.

To keep reviews running without waiting, you can enable usage-based add-on for your organization. This allows additional reviews beyond the hourly cap. Account admins can enable it under billing.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4402a3ca-4d38-4b8b-9d6f-029686c4d8c7

📥 Commits

Reviewing files that changed from the base of the PR and between 3954856 and 519a1e4.

📒 Files selected for processing (2)

• falkordb_bulk_loader/bulk_update.py
• falkordb_bulk_loader/entity_file.py

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/csv-utf8-bom-encoding

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Review rate limit: 0/1 reviews remaining, refill in 47 minutes and 53 seconds.}

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.50%. Comparing base (3954856) to head (519a1e4).

Additional details and impacted files

@@           Coverage Diff           @@
##           master      #69   +/-   ##
=======================================
  Coverage   89.50%   89.50%           
=======================================
  Files           9        9           
  Lines         610      610           
=======================================
  Hits          546      546           
  Misses         64       64           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gkorland]

Copilot Code Review

✅ No significant issues found.

[gkorland]

Agent Review Summary

PR: fix(csv): open input files as utf-8-sig to strip BOM transparently

Overall Assessment: ✅ LGTM — changes are correct, minimal, and well-scoped.

---

What was done

Phase	Action	Result
Code review	Reviewed diff in bulk_update.py and entity_file.py	✅ Correct fix
Completeness check	Audited all open() / io.open() calls in falkordb_bulk_loader/	✅ No remaining read-text opens without encoding=
Rebase	Branch was 5 commits behind master	✅ Rebased and force-pushed
CI	Monitored all checks after rebase push	✅ 7/7 checks passing

---

Code Analysis

The fix is exactly right: switching from implicit-encoding opens to encoding="utf-8-sig" in all three CSV-reading call sites:

• count_entities() in bulk_update.py
• process_update_csv() in bulk_update.py
• EntityFile.__init__() in entity_file.py

The utf-8-sig codec strips a leading UTF-8 BOM (\ufeff) if present and is otherwise identical to utf-8, so there is zero risk of regression for files without a BOM. The root cause (Excel's default "Save as CSV (UTF-8)") is a common real-world issue.

Notes

• No new tests were added for BOM-handling paths. Adding a parametrized unit test that writes a BOM-prefixed CSV and verifies correct column name parsing would strengthen coverage, but is not blocking for this fix.
• The existing test suite passes cleanly on Python 3.10 and 3.11 after the rebase.

---

Reviewed and rebased by GitHub Copilot agent.