-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathDATA_LICENSE
More file actions
50 lines (38 loc) · 2.16 KB
/
Copy pathDATA_LICENSE
File metadata and controls
50 lines (38 loc) · 2.16 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
The data artifact distributed with fojin-cli (fojin-parallels-*.sqlite[.gz])
is licensed CC BY-SA 4.0 — https://creativecommons.org/licenses/by-sa/4.0/
## Source and attribution
Every cross-canon alignment in this artifact comes from Dharmamitra's
MITRA-parallel dataset. None of the alignments are fojin's own.
Alignments: Dharmamitra — https://dharmamitra.org
MITRA-parallel — https://github.com/dharmamitra/mitra-parallel
released CC BY-SA 4.0 (stated by Dharmamitra in that
repository's v1/README.md, "License" section; the repository
has no top-level LICENSE file)
Packaging: fojin.app — https://fojin.app
## Changes made by fojin (CC BY-SA 4.0 §3(a)(1)(B) change indication)
This artifact is an adaptation of MITRA-parallel, not a verbatim copy. The
alignment text itself is unaltered; what fojin added is a processing layer:
- Linked each alignment to its Taishō number (cbeta_id), Chinese title
(title_zh) and fascicle number (juan_num). MITRA-parallel v1 ships four
TSV columns (src_segmentnr, src_original, tgt_segmentnr, tgt_original)
and carries no title or fascicle field.
- Added a normalized query column (zh_norm): 1:1 traditional-to-simplified
character folding plus punctuation stripping, ruleset "t2s-char-1to1-v1".
- Restructured the corpus into SQLite with an FTS5 trigram index, and
dropped rows with an empty zh_text or foreign_text.
Per ShareAlike, this adapted artifact is distributed under the same license.
## Scope
This artifact contains MITRA cross-canon parallels ONLY. fojin's own curated
alignment_pairs are NOT included.
## Citation (requested by the dataset authors, not a license condition)
If you use this data in academic work, Dharmamitra asks that you cite the
MITRA paper:
@article{nehrdich2026mitra,
title={MITRA: A Large-Scale Parallel Corpus and Multilingual Pretrained
Language Model for Machine Translation and Semantic Retrieval for
Pāli, Sanskrit, Buddhist Chinese, and Tibetan},
author={Sebastian Nehrdich and Kurt Keutzer},
journal={arXiv preprint},
year={2026},
url={https://arxiv.org/pdf/2601.06400}
}