CSV Files

CSV v1.0.0

Diff since v0.10.17

CSV.jl 1.0.0

CSV.jl 1.0 rewrites the parsing and writing internals while keeping the main CSV.File, CSV.read, CSV.Rows, CSV.Chunks, CSV.write, and CSV.RowWriter entry points.

Highlights

• A shared, quote-aware structural index supports parallel typed parsing, eager tables, row views, lazy access, and batches.
• New CSV.lazy indexes a source once and parses cells on access. CSV.File(lazyfile) reuses the index.
• Tables.Scan pushes column selection, renaming, conversion, filtering, offset, and limit into the reader.
• CSV.problems(file) exposes structured parse problems. on_error selects summary warnings, silent collection, or a CSV.ParseError.
• The writer supports deterministic parallel output, explicit quote styles, Printf-style float formatting, automatic gzip suffix detection, and parallel gzip and partition output.
• Faster string, time, and floating-point writing reduces formatting overhead and temporary allocations.
• Julia trim compilation is covered by compile-and-execute tests.

Breaking changes

• Julia 1.10 or later is required. Use CSV.jl 0.10 for older Julia versions.
• Inferred text uses DataStrings.DataString. It is an AbstractString; eager columns own their text bytes. Use stringtype=String, or load InlineStrings and request an explicit type, when a consumer requires a particular representation. PosLenString output is retired.
• Pooling is disabled by default. Pass pool=(0.2, 500) to restore the old policy.
• Unquoted empty fields are always missing. Quote a field to represent present empty text. The writer preserves this distinction.
• Date-time inference preserves fractional precision. It normally returns Timestamp{Nanosecond} from Durations.jl instead of Dates.DateTime. Wider date ranges use an exact microsecond representation when possible, otherwise text. Explicit types=DateTime reports sub-millisecond values as parse problems rather than truncating them.
• Default Boolean spellings are true, True, TRUE, false, False, and FALSE. Set truestrings and falsestrings for other spellings.
• Parse recovery records structured problems and emits one summary warning. Use on_error=:collect instead of silencewarnings=true, or on_error=:error to fail fast. Long rows report extra fields rather than widening the schema.
• Some old options and APIs are removed or replaced. types, select, drop, and pool no longer accept functions. CSV.detect is removed; use delim=nothing. See the migration guide for the complete option mapping.
• Source-memory behavior changes. CSV.Rows, CSV.Chunks, and CSV.lazy retain source bytes and a complete structural index. Gzip and non-file sources are fully buffered. These interfaces reduce value materialization but do not provide unbounded stream processing.
• Writer behavior is stricter. An IO is written at its current position. .gz paths enable gzip by default; use compress=:none to disable it. Unknown writer keywords are errors, and quotestyle=:none rejects values that need structural quoting.

Migration guide and upgrade examples · Full release notes

Merged pull requests:

• Rewrite CSV.jl internals for 1.0 (#1196) (@quinnj)
• Improve vector input parsing performance (#1198) (@quinnj)
• Fix chunk detection with short row samples (#1200) (@quinnj)

Closed issues:

• 10-20x slowdown reading multiple files via CSV.read(::Vector{String}, DataFrame) (#1197)
• Regression in 0.10.17: CSV.Chunks fails on well-formed files when chunks hold fewer than rows_to_check rows (#1199)