Feature Request: Automatic saving of metadata with arrow files #3480
Description
It would be great if a DataFrame and it's metadata could be saved to a single file. I believe that arrow supports this.
Right now metadata is not automatically saved when a dataframe is saved as an arrow file. I believe a PR was opened but it appears stalled.
It would be great to have this functionality.
current status of saving to arrow with metadata
using Arrow
using DataFrames
df = DataFrame(a = 1:3, b= 'A':'C')
Arrow.write("test.arrow", df)
df = DataFrame(Arrow.Table("test.arrow"))
colmetadata!(df, :a, "test", "hope this works"; style = :note)
colmetadata(df, :a, "test")
Arrow.write("test2.arrow", df)
df = DataFrame(Arrow.Table("test2.arrow"))
colmetadata(df, :a, "test")
ERROR: ArgumentError: no column-level metadata found for column "a"
Stacktrace:
[1] colmetadata(df::DataFrame, col::Symbol, key::String, default::DataFrames.MetadataMissingDefault; style::Bool)
@ DataFrames ~/.julia/packages/DataFrames/kcA9R/src/other/metadata.jl:367
[2] colmetadata
@ ~/.julia/packages/DataFrames/kcA9R/src/other/metadata.jl:360 [inlined]
[3] colmetadata(df::DataFrame, col::Symbol, key::String)
@ DataFrames ~/.julia/packages/DataFrames/kcA9R/src/other/metadata.jl:360
[4] top-level scope
@ ~/Documents/GitHub/ItsLivePlayground.jl/src/RiverTest.jl:41