[feat] Add Qwen generative recommendation model

[WhiteSwan1]

Summary

• Add a reusable GenerativeRecLM base and a Qwen2RecLM implementation for teacher-forced training and beam-search inference.
• Extend the Qwen tokenizer with per-level SID atoms. Dataset codes stay local 0-based values, while the model derives and applies level_offsets during tokenization and removes them after generation.
• Add ALGR-style escalating beam search, cold-start Hugging Face weight initialization, DCP checkpoint metadata, and DCP-to-HF export support.
• Register the model and export configuration in protobuf, add a linear-decay learning-rate scheduler, and cover SID mapping, prompt splicing, generation, and beam behavior with unit tests.

Compatibility and configuration

• History rows must contain complete items in level order; each code must be in [0, codebook[level]). Data producers must not pre-apply level offsets.
• The answer is read from data_config.label_field, while history is read from the single JAGGED_SEQUENCE feature group.
• Hugging Face export is opt-in through export_config.export_format = HF; the existing TorchScript path remains the default.
• Add transformers 4.51.2 as the pinned runtime backbone dependency.

Testing

• PYTHONPATH=. python -m unittest tzrec.models.generative_rec_lm_test tzrec.models.qwen2_rec_lm_test (37 tests passed)
• ruff check on the four modified model and test modules
• ruff format --check on the same modules
• git diff --check
• bash scripts/gen_proto.sh

[github-actions]

Static review only; no tests or builds were run.

I left inline comments for the correctness and scalability issues. Two general documentation items also need alignment before merge:

• The PR description says SID history/labels are 1-based in [1, codebook[level]], but the implementation, proto, and tests enforce 0-based values in [0, codebook[level]). Producers following the current description will shift or reject every boundary value.
• This adds a public model, feature type, and HF export format without user-facing docs or a runnable example. Please document the data contract, JAGGED_SEQUENCE/prompt requirements, beam/output shape, pretrained loading, and HF export workflow.

[WhiteSwan1]

Thanks for github actions — all seven reproduce. Four fixed in 07443e3 and 17a4631, two I'd like to leave, one deferred, docs separate.

Fixed

1. HF export bypasses dense EMA (07443e3) — correct, and a regression from the recent upstream merge. dcp_to_hf reads <checkpoint>/model while EMA lives in <checkpoint>/dense_ema, so TorchScript export shipped averaged weights and HF export shipped raw ones. I chose to reject the combination rather than overlay: the converter never builds a model, so applying EMA there would duplicate restore_model's mapping logic. Test added.

3. value_dim unsupported (17a4631) — agreed, _parse assumes one code per position. Rejected at construction now. The field has to stay; BaseFeature reads it unconditionally.

5. Stale FG comment in the proto (17a4631) — correct, it predates the fg passthrough. Fixed.

PR description says 1-based — fixed; the code has been 0-based since that switch.

Not changing

2. num_return_sequences vs. the capped width — real, but the final width only shrinks when beam_widths[-1] > effective[-2] * codebook[-1]. At codebook 512 or 8192 that needs a final width above the codebook itself, which will not fit in memory; even the repo's [4,4,4] mock at [4,8,16] is uncapped. Reaching it takes a codebook around 4 or a decreasing schedule. Intermediate capping is harmless — later levels multiply the width back up — and the failure returns fewer candidates, not wrong ones. The existing check catches the mistake a user actually makes.

4. Non-integral / non-finite codes — the realistic route to NaN is a missing value, and that is already rejected before _parse: pyarrow raises ArrowInvalid: ... 1 nulls, but zero_copy_only was True. So NaN needs a float column with NaN written into it deliberately. The fg point is right that _fg_json declares a float type, but float32 is exact to 2^24 against a largest codebook of 8192 — floats by dtype, never by magnitude.

Deferred

6/7. Beam KV duplication and full-vocab fp32 logits — both accurate. Context the static pass lacked: the band-slice form already cut the sustained score tensor 538 MiB → 25 MiB and level-3 topk 499 ms → 69 ms, and [100, 200, 400] runs complete on H20s. Paged cache indirection and a fused chunked reduction are substantial changes against a kernel verified equivalent to HF beam search across 96 configurations, so I'd rather file them as follow-ups with that profile than take them here.

Docs

Agreed. Landing as a separate PR covering the sample contract, the JAGGED_SEQUENCE group rule, prompt_template, the beam knobs, cold-start loading, and export_format: HF.