Skip to content

Commit dfc0edc

Browse files
cburgarddpiparo
authored andcommitted
[RF][HS3] Bring the HS3 import and export closer to the HS3 standard
Fix several deviations of RooFit's HS3 layer from the HS3 standard (hep-statistics-serialization-standard master at cb3e9eb). None of the changes depend on pending HS3 PRs; the ones that do follow in #23840. Stricter checks only apply to documents that declare a recent enough hs3_version, so files written by older ROOT versions remain readable. Metadata and names: * hs3_version must be a string. Its MAJOR.MINOR is parsed, and a warning is printed if it can't be parsed or is newer than what RooFit writes. Importers can query it with RooJSONFactoryWSTool::hs3VersionInput(). * Duplicate names of top-level objects are an error from HS3 0.3 on, and only a warning for older documents. * On export, likelihoods are named "<analysis>_likelihood", analyses are named after their ModelConfig, and transformed helper functions that several pdfs share are written only once. Import: * The "default_domain" is optional. Without it, variables get the range of the first domain that lists them. * Parameters that only appear in a parameter point are created. They are constant if "const" is true or if no domain lists them. A string value sets the label of a category. Index categories of combined distributions are now created before the parameter points. * Observables of unbinned data without a domain range take their range from the "min"/"max" of the data axes. Infinite bounds (null) are supported. * "init" of an analysis is restored as the ModelConfig snapshot under the same name. A missing parameter point is reported. * An explicit "extended" flag of mixture_dist is checked against the number of coefficients. * Malformed input is rejected instead of being read out of bounds: covariance and correlation matrices must be square, "errors" must have as many entries as "contents", and "edges" must contain at least two strictly ascending values. * Errors of zero in binned data stay zero. Before, they could be read as squared weight = content. Export: * Axes of unbinned data only carry "name", "min" and "max". * The ModelConfig snapshot is written as "init" of the analysis, which no longer has the side effect of loading the snapshot values. * Ternaries in formulas are written as ifelse(c, a, b), and ifelse() is translated back to a ternary on import. Type keys (the old keys are still imported, with a deprecation warning): * ARGUS_dist -> argus_dist * binsampling -> binsampling_dist. Before, RooFit could not read back its own export. * rate_extended_dist uses the fields "dist" and "rate" instead of "pdf" and "norm". HistFactory: * "LogNormal" is accepted as a constraint type, alongside "Lognormal". Log-normal constraints of gamma parameters are rejected instead of silently becoming Gaussian. * The parameter of a normfactor is named after "parameter". "name" is only the label of the modifier. * normsys/histosys modifiers without constraint are unconstrained from HS3 0.2 on, with one warning per channel. Older documents, and objects imported with importJSONElement(), keep the implicit unit Gaussian. On export, unconstrained modifiers are allowed. A normsys with a function as parameter is refused. * The staterror constraint type is read from the modifier ("constraint", or "constraint_type" as written by ROOT 6.38 and 6.40.00), with the channel-level setting as fallback. It is written in both places, so Gaussian stat errors are no longer lost in a round trip. * A modifier without interpolation and without "default_interpolation" is an error from HS3 0.3 on. Older documents keep code 4, with a deprecation warning. * Fix the export of normsys modifiers recognized from formulas like 1+0.1*alpha: hi/lo were written as 0.1/-0.1 instead of the absolute factors 1.1/0.9, and the constraint was dropped. Other: * Warnings of RooJSONFactoryWSTool are logged on the ObjectHandling topic. Before, they went to the IO topic, which no default stream shows. * Add ModelConfig::GetSnapshotName(). * Update the RooFitHS3 developer documentation. * Make the constraints and interpolation of the rf515_hfJSON tutorial explicit. The fit result is unchanged. Files written with this version use the new type keys and ifelse(), which ROOT 6.40 and older cannot read. Assisted-by: claude-code:claude-opus-5-5 (cherry picked from commit d61fecf)
1 parent 72d64b3 commit dfc0edc

13 files changed

Lines changed: 2108 additions & 246 deletions

‎roofit/doc/developers/roofit_hs3.md‎

Lines changed: 51 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -47,12 +47,16 @@ implementation(s) take precedence.
4747
### Writing your own importers and exporters: Proxy-based
4848

4949
Proxy-based implementations can be added very easily and without
50-
actually writing any `C++` code -- you only need to add a short item
51-
to a list in a `JSON` file, namely the
52-
[export keys](https://github.com/root-project/root/blob/master/etc/RooFitHS3_wsexportkeys.json)
50+
actually writing any `C++` logic -- you only need to add a short item
51+
to a `JSON` catalog, namely the
52+
[export keys](https://github.com/root-project/root/blob/master/roofit/hs3/src/RooFitHS3_wsexportkeys.cxx)
5353
for an exporter, or the
54-
[factory expressions](https://github.com/root-project/root/blob/master/etc/RooFitHS3_wsfactoryexpressions.json)
55-
for an importer.
54+
[factory expressions](https://github.com/root-project/root/blob/master/roofit/hs3/src/RooFitHS3_wsfactoryexpressions.cxx)
55+
for an importer. The built-in catalogs are `JSON` documents embedded as
56+
raw string literals in these two source files. They are compiled into
57+
the `RooFitHS3` library and loaded automatically when the library is
58+
loaded, so there are no separate `JSON` files to install or find at
59+
runtime.
5660

5761
This works in the following way: Every `RooFit` class performs
5862
dependency tracking via proxies, which have names. This can be
@@ -67,7 +71,7 @@ interface. Hence:
6771
expression`.
6872

6973
For the importer, an entry in the
70-
[factory expressions](https://github.com/root-project/root/blob/master/etc/RooFitHS3_wsfactoryexpressions.json)
74+
[factory expressions](https://github.com/root-project/root/blob/master/roofit/hs3/src/RooFitHS3_wsfactoryexpressions.cxx)
7175
needs to be added as follows:
7276

7377
``` {.json}
@@ -82,7 +86,7 @@ needs to be added as follows:
8286
```
8387

8488
Similarly, for the exporter, an entry in the
85-
[export keys](https://github.com/root-project/root/blob/master/etc/RooFitHS3_wsexportkeys.json)
89+
[export keys](https://github.com/root-project/root/blob/master/roofit/hs3/src/RooFitHS3_wsexportkeys.cxx)
8690
needs to be added as follows:
8791

8892
``` {.json}
@@ -97,12 +101,18 @@ needs to be added as follows:
97101
```
98102

99103

100-
If you don't want to edit the central `json` files containing the
101-
factory expressions or export keys, you can also put your custom
102-
export keys or factory expressions into a different json file and load
103-
that using `RooFit::JSONIO::loadExportKeys(const std::string
104+
To contribute a catalog entry to `ROOT`, edit the embedded catalog in
105+
`roofit/hs3/src/RooFitHS3_wsexportkeys.cxx` or
106+
`roofit/hs3/src/RooFitHS3_wsfactoryexpressions.cxx` and rebuild `ROOT`.
107+
If you don't want to change `ROOT` itself, you can put your custom
108+
export keys or factory expressions into your own `JSON` file (with the
109+
same format as the embedded catalogs) and add them to the built-in ones
110+
at runtime using `RooFit::JSONIO::loadExportKeys(const std::string
104111
&fname)` and `RooFit::JSONIO::loadFactoryExpressions(const
105-
std::string &fname)`.
112+
std::string &fname)`. Both functions also have an overload that reads
113+
from a `std::istream`. You can inspect the currently known entries with
114+
`RooFit::JSONIO::printExportKeys()` and
115+
`RooFit::JSONIO::printFactoryExpressions()`.
106116

107117
If either the importer or the exporter cannot be created with factory
108118
expressions and export keys, you can instead write a custom `C++`
@@ -121,9 +131,11 @@ in `ROOT`.
121131
Any importer should take the following form:
122132

123133
``` {.cpp}
134+
using RooFit::Detail::JSONNode;
135+
124136
class MyClassFactory : public RooFit::JSONIO::Importer {
125137
public:
126-
bool importFunction(RooJSONFactoryWSTool *tool, const JSONNode &p) const override
138+
bool importArg(RooJSONFactoryWSTool *tool, const JSONNode &p) const override
127139
{
128140
std::string name(RooJSONFactoryWSTool::name(p));
129141
@@ -145,14 +157,24 @@ Any importer should take the following form:
145157
};
146158
```
147159

148-
If the class you are trying to import inherits from `RooAbsPdf` rather
149-
than from `RooAbsReal`, you should define `importPdf` instead of
150-
`importFunction`, with the same signature.
160+
The same `importArg` function is used for functions (inheriting from
161+
`RooAbsReal`) and pdfs (inheriting from `RooAbsPdf`). If the importer
162+
can't handle a given `JSON` node, it should return `false` so that the
163+
next importer is tried. The functions `importFunction` and `importPdf`
164+
of older `ROOT` versions are still called for backwards compatibility,
165+
so existing importers that override them keep working, but new code
166+
should override `importArg`.
151167

152168
Once your importer implementation exists, you need to register it with the tool using a line like the following:
153169

154170
``` {.cpp}
155-
RooFit::JSONIO::registerImporter("<json key>", new MyClassFactory(), true);
171+
RooFit::JSONIO::registerImporter<MyClassFactory>("<json key>", true);
172+
```
173+
174+
This is a shorthand for passing an instance owned by a `std::unique_ptr`:
175+
176+
``` {.cpp}
177+
RooFit::JSONIO::registerImporter("<json key>", std::make_unique<MyClassFactory>(), true);
156178
```
157179

158180
As there can be several importers for the same `json` key, the last
@@ -189,9 +211,20 @@ The implementation of an exporter works in a very similar fashion:
189211
Also this needs to be registered with the tool
190212

191213
``` {.cpp}
192-
RooFit::JSONIO::registerExporter(MyClass::Class(), new MyClassStreamer(), true);
214+
RooFit::JSONIO::registerExporter<MyClassStreamer>(MyClass::Class(), true);
193215
```
194216

217+
or equivalently
218+
219+
``` {.cpp}
220+
RooFit::JSONIO::registerExporter(MyClass::Class(), std::make_unique<MyClassStreamer>(), true);
221+
```
222+
223+
If the `TClass` of your class is not available where you register the
224+
exporter, you can also pass the class name as a string instead of
225+
`MyClass::Class()`. If `exportObject` returns `false`, the next
226+
exporter is tried, and finally the export keys.
227+
195228
For more complicated cases where members are lists of elements, the
196229
methods `is_seq()`, `set_seq()`, `is_map()`, `set_map()` and
197230
ranged-based for-loops via `children()` might come in handy.

‎roofit/hs3/inc/RooFitHS3/RooJSONFactoryWSTool.h‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,9 @@ class RooJSONFactoryWSTool {
206206
std::string getStringAttribute(const std::string &obj, const std::string &attrib);
207207
void setStringAttribute(const std::string &obj, const std::string &attrib, const std::string &value);
208208

209+
/// The "hs3_version" of the document that is currently being imported, or an empty string outside of an import.
210+
std::string const &hs3VersionInput() const { return _hs3VersionInput; }
211+
209212
private:
210213
template <class T>
211214
T *requestImpl(const std::string &objname);
@@ -251,6 +254,7 @@ class RooJSONFactoryWSTool {
251254
const RooFit::Detail::JSONNode *_attributesNode = nullptr;
252255
RooFit::Detail::JSONNode *_rootnodeOutput = nullptr;
253256
RooFit::Detail::JSONNode *_varsNode = nullptr;
257+
std::string _hs3VersionInput;
254258
RooWorkspace &_workspace;
255259

256260
// objects to represent intermediate information

‎roofit/hs3/src/Domains.cxx‎

Lines changed: 16 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@
1111
*/
1212

1313
#include "Domains.h"
14+
#include "JSONIOUtils.h"
1415

1516
#include <RooAbsBinning.h>
1617
#include <RooBinning.h>
@@ -29,15 +30,6 @@ constexpr static auto defaultDomainName = "default_domain";
2930

3031
namespace {
3132

32-
double readBound(RooFit::Detail::JSONNode const &node, const char *key, double defaultValue)
33-
{
34-
if (!node.has_child(key)) {
35-
return defaultValue;
36-
}
37-
auto const &bound = node[key];
38-
return bound.is_null() ? defaultValue : bound.val_double();
39-
}
40-
4133
void writeBound(RooFit::Detail::JSONNode &node, double value)
4234
{
4335
if (RooNumber::isInfinite(value)) {
@@ -54,6 +46,12 @@ void Domains::populate(RooWorkspace &ws) const
5446
auto default_domain = _map.find(defaultDomainName);
5547
if (default_domain != _map.end()) {
5648
default_domain->second.populate(ws);
49+
} else {
50+
// Without the RooFit-specific "default_domain", the variables are created with the range of the domain that
51+
// mentions them.
52+
for (const auto &domain : _map) {
53+
domain.second.populate(ws);
54+
}
5755
}
5856
for (const auto &domain : _map) {
5957
if (domain.first == defaultDomainName)
@@ -78,15 +76,20 @@ void Domains::writeVariable(RooRealVar &var) const
7876
auto default_domain = _map.find(defaultDomainName);
7977
if (default_domain != _map.end()) {
8078
default_domain->second.writeVariable(var);
79+
return;
80+
}
81+
// Without the RooFit-specific "default_domain", use the first domain that mentions the variable.
82+
for (const auto &domain : _map) {
83+
if (domain.second.hasVariable(var.GetName())) {
84+
domain.second.writeVariable(var);
85+
return;
86+
}
8187
}
8288
}
8389

8490
void Domains::readJSON(RooFit::Detail::JSONNode const &node)
8591
{
86-
auto default_domain_element = RooJSONFactoryWSTool::findNamedChild(node, defaultDomainName);
87-
if (!default_domain_element) {
88-
RooJSONFactoryWSTool::error("\"domains\" do not contain \"" + std::string{defaultDomainName} + "\"");
89-
}
92+
// The "default_domain" is written by RooFit, but it is not required by the HS3 standard.
9093
for (auto &domain : node.children()) {
9194
if (!domain.has_child("name")) {
9295
RooJSONFactoryWSTool::error("encountered domain without \"name\"");

0 commit comments

Comments
 (0)