Problem in converting FITS file tables to DataFrames

[stefanocovino]

Dear friends,

Recently, I can't import tables stored ad FITS files as DataFrame since I see errors never met before. I say "before" since it worked till a few months ago. I am using julia 11.3, on a Debian linux.

Thanks a lot,
Stefano

---

I do very simple things, e.g.:

using FITSIO
using DataFrames

hdu = FITS("NGC_2682.fits")
File: NGC_2682.fits
Mode: "r" (read-only)
HDUs: Num Name Type
1 Image
2 Table

hdu[2]
File: NGC_2682.fits
HDU: 2
Type: Table
Rows: 1585
Columns: Name Size Type TFORM
gaia_dr2_source.designation String 40A
gaia_dr2_source.source_id Int64 K
gaia_dr2_source.ra Float64 D
gaia_dr2_source.dec Float64 D
gaia_dr2_source.parallax Float64 D
gaia_dr2_source.parallax_error Float64 D

many more lines....

data = DataFrame(hdu[2])

[2533] signal 11 (1): Segmentation fault
in expression starting at REPL[5]:1
jl_datatype_layout at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:1338 [inlined]
gc_mark_outrefs at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2771 [inlined]
gc_mark_loop_serial_ at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2943
gc_mark_loop_serial at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:2966
gc_mark_loop at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3148 [inlined]
jl_gc_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3537
ijl_gc_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3898
maybe_collect at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:922 [inlined]
jl_gc_pool_alloc_inner at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:1324
jl_gc_pool_alloc_noinline at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:1391 [inlined]
jl_gc_alloc at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia_internal.h:523 [inlined]
jl_gc_alloc at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gc.c:3951
new_genericmemory at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/genericmemory.c:56 [inlined]
jl_alloc_genericmemory at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/genericmemory.c:99
GenericMemory at ./boot.jl:516 [inlined]
array_new_memory at ./array.jl:1058 [inlined]
#134 at ./array.jl:1129
_growend! at ./array.jl:1116 [inlined]
push! at ./array.jl:1261 [inlined]
compute_domtree_nodes! at ./compiler/ssair/domtree.jl:270
update_domtree! at ./compiler/ssair/domtree.jl:260 [inlined]
construct_domtree at ./compiler/ssair/domtree.jl:242
CFGTransformState! at ./compiler/ssair/ir.jl:654
IncrementalCompact at ./compiler/ssair/ir.jl:724 [inlined]
adce_pass! at ./compiler/ssair/passes.jl:2016
run_passes_ipo_safe at ./compiler/optimize.jl:994
run_passes_ipo_safe at ./compiler/optimize.jl:1009 [inlined]
optimize at ./compiler/optimize.jl:983
jfptr_optimize_42723.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
finish_nocycle at ./compiler/typeinfer.jl:265
_typeinf at ./compiler/typeinfer.jl:249
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_edge at ./compiler/typeinfer.jl:923
abstract_call_method at ./compiler/abstractinterpretation.jl:660
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:102
abstract_call_known at ./compiler/abstractinterpretation.jl:2200
abstract_call at ./compiler/abstractinterpretation.jl:2282
abstract_call at ./compiler/abstractinterpretation.jl:2275
abstract_call at ./compiler/abstractinterpretation.jl:2420
abstract_eval_call at ./compiler/abstractinterpretation.jl:2435
abstract_eval_statement_expr at ./compiler/abstractinterpretation.jl:2451
abstract_eval_statement at ./compiler/abstractinterpretation.jl:2749
abstract_eval_basic_statement at ./compiler/abstractinterpretation.jl:3065
typeinf_local at ./compiler/abstractinterpretation.jl:3319
typeinf_nocycle at ./compiler/abstractinterpretation.jl:3401
_typeinf at ./compiler/typeinfer.jl:244
typeinf at ./compiler/typeinfer.jl:215
typeinf_ext at ./compiler/typeinfer.jl:1101
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1139
typeinf_ext_toplevel at ./compiler/typeinfer.jl:1135
jfptr_typeinf_ext_toplevel_39915.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
jl_type_infer at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:390
jl_generate_fptr_impl at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jitlayers.cpp:519
jl_compile_method_internal at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2536 [inlined]
jl_compile_method_internal at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2423
_jl_invoke at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:2940 [inlined]
ijl_apply_generic at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/gf.c:3125
#52 at ./none:0
iterate at ./generator.jl:48 [inlined]
collect at ./array.jl:791
unknown function (ip: 0x7f63b531df82)
getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582
#fromcolumns#912 at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:36
fromcolumns at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:36 [inlined]
#DataFrame#914 at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:59 [inlined]
DataFrame at /home/stefanocovino/.julia/packages/DataFrames/kcA9R/src/other/tables.jl:48
unknown function (ip: 0x7f63b531ae52)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
do_call at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:126
eval_value at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:223
eval_stmt_value at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:174 [inlined]
eval_body at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:663
jl_interpret_toplevel_thunk at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/interpreter.c:821
jl_toplevel_eval_flex at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:943
jl_toplevel_eval_flex at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:886
ijl_toplevel_eval_in at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/toplevel.c:994
eval at ./boot.jl:430 [inlined]
eval_user_input at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:245
repl_backend_loop at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:342
#start_repl_backend#59 at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:327
start_repl_backend at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:324
#run_repl#72 at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:483
run_repl at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/usr/share/julia/stdlib/v1.11/REPL/src/REPL.jl:469
jfptr_run_repl_10097.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/share/julia/compiled/v1.11/REPL/u0gqU_XvZAg.so (unknown line)
#1150 at ./client.jl:446
jfptr_YY.1150_14693.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/share/julia/compiled/v1.11/REPL/u0gqU_XvZAg.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
jl_f__call_latest at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/builtins.c:875
#invokelatest#2 at ./essentials.jl:1055 [inlined]
invokelatest at ./essentials.jl:1052 [inlined]
run_main_repl at ./client.jl:430
repl_main at ./client.jl:567 [inlined]
_start at ./client.jl:541
jfptr__start_73609.1 at /home/stefanocovino/.julia/juliaup/julia-1.11.3+0.x64.linux.gnu/lib/julia/sys.so (unknown line)
jl_apply at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/julia.h:2157 [inlined]
true_main at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jlapi.c:900
jl_repl_entrypoint at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/src/jlapi.c:1059
main at /cache/build/builder-demeter6-3/julialang/julia-release-1-dot-11/cli/loader_exe.c:58
unknown function (ip: 0x7f63c21aa249)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x4010b8)
Allocations: 3994572 (Pool: 3994431; Big: 141); GC: 6
Segmentation fault (core dumped)

[bkamins]

The problem seems to be originating from this line:

getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582

which is in the FITSIO.jl package.

[stefanocovino]

I see. Thanks a lot for the analysis. Just to know, should I open an issue with FITSIO or is this something you take care? Stefano Mobilis in mobile Il giorno sab 15 feb 2025 alle 10:11 Bogumił Kamiński < ***@***.***> ha scritto:

…

The problem seems to be originating from this line: getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582 which is in the FITSIO.jl package. — Reply to this email directly, view it on GitHub <#3489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVYHEKN67FFTZ23XD3MT32P4AD3AVCNFSM6AAAAABXFAVGC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHAZTCNRRGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***> [image: bkamins]*bkamins* left a comment (JuliaData/DataFrames.jl#3489) <#3489 (comment)> The problem seems to be originating from this line: getcolumn at /home/stefanocovino/.julia/packages/FITSIO/VqV3x/src/table.jl:582 which is in the FITSIO.jl package. — Reply to this email directly, view it on GitHub <#3489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVYHEKN67FFTZ23XD3MT32P4AD3AVCNFSM6AAAAABXFAVGC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHAZTCNRRGQ> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bkamins]

I think it is better if you open the issue there. I do not know this package and have never used it.

[stefanocovino]

Ok, thanks. Stefano Il giorno sab 15 feb 2025 alle 15:34 Bogumił Kamiński < ***@***.***> ha scritto:

…

I think it is better if you open the issue there. I do not know this package and have never used it. — Reply to this email directly, view it on GitHub <#3489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVYHDHV6LFL62567YSOW32P5GAJAVCNFSM6AAAAABXFAVGC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHE2DSNZXGM> . You are receiving this because you authored the thread.Message ID: ***@***.***> [image: bkamins]*bkamins* left a comment (JuliaData/DataFrames.jl#3489) <#3489 (comment)> I think it is better if you open the issue there. I do not know this package and have never used it. — Reply to this email directly, view it on GitHub <#3489 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAGVYHDHV6LFL62567YSOW32P5GAJAVCNFSM6AAAAABXFAVGC6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRQHE2DSNZXGM> . You are receiving this because you authored the thread.Message ID: ***@***.***>