Fixing garbled characters in CSV files

What garbled text actually is

When a CSV opens and shows character salad instead of readable text — café where you expected café, or a run of unrelated symbols where you expected Japanese — the data is not damaged. The bytes inside the file are exactly as they were; the program reading them is interpreting them as a different encoding.

The same text, saved as UTF-8 and then opened as Shift-JIS, comes out as nonsense, and vice versa. Garbled text is therefore a mismatch between the encoding used to write the file and the one used to read it, and reopening with the correct encoding restores it completely.

The two encodings behind Japanese CSV

A BOM (byte order mark) is a marker at the start of a UTF-8 file, and with one present Excel can recognise the file as UTF-8. So if the file is going to be opened in Excel, UTF-8 with BOM — Excel's "CSV UTF-8" option — is the safe choice.

Identifying the cause

1. Open the file in Notepad (or VS Code)Editors normally detect the encoding automatically. If the text looks correct there, the problem is on the reading side — Excel's assumption.
2. If the editor shows garbage too, reopen with a different encodingIn VS Code, use Reopen with Encoding and try Shift-JIS, then UTF-8. Whichever one displays correctly is the file's real encoding.
3. Check whether you have already saved over the garbled fileIf you have, skip to When it cannot be fixed. If not, carry on with the repair steps below.

Fix 1: reopen it through Excel's import dialog (recommended)

1. Do not double-click the CSV — open a blank workbook in Excel instead
2. Choose Data → From Text/CSV and pick the file
3. Switch the encoding under File Origin in the previewUse 65001 for UTF-8 and 932 for Shift-JIS. Once the preview shows readable text, you have the right one.
4. Click LoadThe data lands in the sheet already displaying correctly.

Because this approach never writes to the file, it is the safest of the three.

Fix 2: convert the encoding and save a copy

Here you convert the file itself to whatever encoding the recipient needs. For a single file, Notepad is enough.

1. Open the file in NotepadThis assumes it is displaying correctly in whatever encoding Notepad picks.
2. Choose Save As and change the EncodingPick ANSI (which means Shift-JIS) or UTF-8, then save.
3. Open it in the target system to confirmAlways run an acceptance check after converting.

If you need to convert a whole folder of files, the command line is faster.

# Convert every .csv in the folder to UTF-8 (with BOM)
Get-ChildItem *.csv | ForEach-Object {
    $t = Get-Content $_.FullName -Encoding Default
    $t | Set-Content $_.FullName -Encoding UTF8
}

Fix 3: specify the encoding in code

When code reads the file, stating the encoding every single time is the only real fix. For Japanese CSV files, four values cover essentially every case.

import pandas as pd

# Read a CSV saved by Excel (no need to worry about the BOM)
df = pd.read_csv("data.csv", encoding="utf-8-sig")
# CSV from Japanese business software
df2 = pd.read_csv("data2.csv", encoding="cp932")

If you run into an error, see the encoding cheat sheet on the reading and writing guide.

Rules that stop it happening again

1. Agree the encoding at every handoverWrite it down rather than leaving it verbal: "Shift-JIS for external submissions, UTF-8 for tool integrations".
2. Put the encoding in the file nameA name like "sales_2026_utf8.csv" tells the recipient what they are dealing with.
3. Standardise on "CSV UTF-8" when you export from ExcelDrop to Shift-JIS for a specific recipient only when you have actually seen it break in their environment.
4. Always convert a copyNever overwrite the original file.

When it cannot be fixed (you already saved over it)

Your options:

• Re-obtain the data from its source — the system that exported it, or the person who sent it
• Look for the original in email attachments (Outlook and similar), shared-folder version history, or your downloads folder
• In a few narrow cases (a UTF-8 → Shift-JIS → UTF-8 round trip, for example) a manual repair tool can help, but do not count on it

"Getting back to the state before I overwrote it" is not an encoding technique — it is a backup problem. Build the habit of copying any important CSV the moment it arrives.

Frequently asked questions