The desire to support weighted aggregations (combined with my decision to stick with DataFrames, which are fine except for this one thing) led me down an ugly path of sticking big vectors of data in column metadata, cluttering up the aggregation code. Ultimately I don't think we need to be storing this data in the object we give back to the user at all — the initial aim was to support re-aggregation, but we're not trying to support that anymore — we can store it as real columns and get rid of it as soon as aggregation is done.
The desire to support weighted aggregations (combined with my decision to stick with
DataFrames, which are fine except for this one thing) led me down an ugly path of sticking big vectors of data in column metadata, cluttering up the aggregation code. Ultimately I don't think we need to be storing this data in the object we give back to the user at all — the initial aim was to support re-aggregation, but we're not trying to support that anymore — we can store it as real columns and get rid of it as soon as aggregation is done.