Questions regarding Spirit maintenance

[saki7]

I’ve been endorsed as the new maintainer of Spirit by @djowel!
See this thread for context: #800 (comment)

@djowel, I’d really appreciate your feedback on the items listed below. Would it be reasonable to assume that I, as a maintainer, can move forward with the actions described here?

cc: @Kojoley

---

Gaining Access to the GitHub Organization

What’s the process for obtaining maintainer access to the relevant repositories?
If I understand @djowel’s intent correctly, I should be granted access to Spirit along with the accompanying libraries.

Deprecation and removal of Spirit.Classic

Documenting "Classic is now deprecated." is easy. However, in practice, that won’t meaningfully change how the codebase is used. As long as the classic headers remains in the repo, users will continue to rely on it, just as they have for the past 15 years.

So my plan is to first document the deprecation clearly in the official docs. Then, after a few major Boost releases, we should proceed to remove the Classic codebase entirely.

Deprecation of Spirit.Qi

I understand Qi is still used in many production systems and mature codebases. That said, I believe we should also begin deprecating Qi—though more gradually than Classic.

My reasoning:

• I no longer use Qi in new projects.
• The X3 codebase has seen major improvements (thanks to @Kojoley), and the divergence between Qi and X3 has become increasingly difficult to maintain.
• Both @djowel and @Kojoley are less active these days. With limited manpower, maintaining two major parser frameworks isn't sustainable.
• X3 has been available for many years. If we don't deprecate Qi now, it will become the next Classic!

(future) Deprecation of Spirit.Karma

To my understanding, Karma can't be deprecated or removed soon, as there's no alternative implementation. Someone needs to write a brand new serialization system based on the X3 codebase, but this is gonna be an independent matter, I guess. For now, I don't have any immediate plans for this.

Or... should we just remove it and state that "serialization is not supported"?

Deprecation of the accompanying libraries (Fusion, Phoenix, Tuple)

As far as I know, they are mainly used by pre-X3 codebase for metaprogramming. C++23 makes most of this possible natively. Given that, I believe we should also deprecate these libraries.

Migrating documentations

I'm planning to migrate X3's docs to AsciiDoc. It's going to be a substantial PR, and I’d like to avoid the risk of it being rejected due to stylistic preferences or legacy concerns. How should we build consensus for major changes like this? Should I just simply submit the PR?

[saki7]

@djowel, sorry for the long message. If you don’t have time to review all the details, I’d just appreciate maintainer access to the relevant repositories. That way, I can start making practical, well-considered decisions on my own. (I'm always open to feedback, of course.)

[Kojoley]

The main reason I've stopped working on improvements is because I got yelled pretty much every time I fix a bug or change some internal stuff that no one should've relied on. And I get their viewpoint, Boost nowadays is seen as a collection of mostly legacy battle-tested libraries, a project uses Boost because it have used it for a long while and people that maintain it don't want their working code been disturbed.

Have you done a research on consequences of deprecating and removing Spirit.Classic? I also was naïve, though it could be done, until I discovered that not only there are huge open source projects that still use classic version, https://github.com/boostorg/wave is using it.

[saki7]

In short: I will make breaking changes if I believe that's the practical solution.

Well, I'm okay with getting yelled at, if it is the consequence of breaking changes. Someone needs to break things if the root cause is the unmaintained code. When I was an ordinary user, I myself was sometimes one of those people who are yelling at maintainers. But if I am going to be the maintainer, I guess I need to live with that.

Maintainers have the right to deprecate things. People have the right to yell at maintainers. These two facts can exist simultaneously.

In other words: I promise to always be open for any sort of feedback (from both maintainers and community users), but, at the same time, I don't care if people are yelling at me for my realistic and practical decisions that I make as a maintainer. In case if I get mentally destroyed then I'll just step down. But that's not something that would be happening anytime soon. I am just getting started.

Honestly, I'd personally like to thank you for all your works done in recent years.

[pdimov]

That's not how things are supposed to work. If you don't want to maintain Spirit Classic, then don't. Deprecating and removing it is not the same thing. Ideally, it would be split into its own repo, and left to whoever wants to maintain it. (And if nobody wants to do that, then I win that honor by default, because someone has to. Not that I'm thrilled to maintain everything, or even have time to, but there's no realistic alternative.)

And that goes for every Spirit project; all these things need to be separate repositories, with their own willing maintainers, or lack thereof.

[pdimov]

Have you done a research on consequences of deprecating and removing Spirit.Classic?

Boost.Serialization uses it, for one. And is unlikely to be fixed if broken by its removal. (Even if we ignore Quickbook.)

[saki7]

People always have the right to stick to a specific Git commit hash. People always have the right to keep their own tarballs.

Nobody is obliged to keep their now-obsolete code forever. Remember, Boost had actually done it for dozens of years and it is now causing real issues! Seriously, take a look at @Kojoley's comment. Boost itself has proven that such ignorant stance is against the sustainability of an open source project. It's causing real harm to the maintainers.

[saki7]

Well, I honestly didn’t expect the very first "yell" to come from a core developer. Much appreciated. I guess that's how veterans treat newcomers on this community. Very good. I'm still looking forward to hearing from @djowel soon.

[pdimov]

People always have the right to stick to a specific Git commit hash.

No, that's not possible in Boost. Quickbook can't stick to a specific Git commit hash of Spirit. And even if it could, if library X picked a certain Git commit hash and library Y picked a different Git commit hash, library Z wouldn't be able to use X and Y.

[saki7]

Why not bundle quickbook.exe? It should not be required to build the documentation framework from scratch, when you're just building a documentation, in the first place.

[pdimov]

Well, the way it works is we release source, and then the downstream packagers, such as the Debian package maintainers, use that source to compile the libraries and the executables. Even if we released binaries, we couldn't compile the quickbook binary in the way they need it (because we don't know their compiler version and options yet.)

[saki7]

We can build the portable binaries like i386, amd64, windows_x64, limiting the arch related flags to the minimal subset. There are tons of open source frameworks that distribute such binaries. Distributing quickbook on source code format and expecting people to build it from scratch is like bundling the entire CMake source code for your editor's CMake extension.

[pdimov]

I'm sure we could figure something out if it were just Quickbook, doubly so if we just ignore users of Quickbook outside Boost (if it runs on our CI that's probably 95% of it.)

But consider Serialization. If we physically remove Classic, Serialization will fail to build, and who knows how many dependent packages will also fail.

Now the way it works is that Debian, Ubuntu, Fedora ship a certain Boost release in their OS version N, and in version N+1 they try to upgrade the Boost release to a more recent one. So they run their build scripts, and if everything's fine, they ship the newer Boost, and if it isn't, they make a reasonable effort to fix the breakage, and if successful, ship, if not, keep shipping the older release.

So when the more recent Boost causes 9218 packages to fail to build, they are not going to ship it.

[saki7]

Distros are distros. They don't decide how we (boostorg) maintain our own libraries. Of course we care about backwards compatibility, but this matter is not something that can be included in a rather general scope of backwards compatibility. Even OpenSSL and curl sometimes make breaking changes. They tend to care how their library impact the distros, but they don't serve for distros.

[pdimov]

Well, I honestly didn’t expect the very first "yell" to come from a core developer.

That wasn't a "yell", I proposed a way forward that would give you what you want (no Classic in this repo) while avoiding the yells. And in return, you accused me of treating newcomers harshly.

Fair enough, I suppose.

Either way, this conversation properly belongs on the list because the effects of the discussed changes have a much wider impact.

[djowel]

Finally I agreed on that point, I said no warnings are necessary. What are you worrying about?

Not worried about anything. I merely suggested an alternative if you still want to push the deprecation warning idea. If not, then good. Let's move forward.

[saki7]

OK, so what are the next steps we should take?

[djowel]

OK, so what are the next steps we should take?

Wait for @pdimov

[pdimov]

I think we need to summarize the actual steps at least (or kindly ask Peter to guide us). I still have no idea how the headers should be placed / how the build scripts should be modified in order to make such transition.

What we need to do is:

• copy boostorg/spirit into boostorg/spirit_classic
• add boostorg/spirit_classic as a Boost submodule
• delete classic/ and the classic headers from boostorg/spirit
• delete everything else from boostorg/spirit_classic
• move classic/ to / in boostorg/spirit_classic
• add build.jam, CMakeLists.txt to spirit_classic
• adjust CMakeLists.txt files of libraries depending on Classic to reference the new CMake target
• and probably other things in boostorg/spirit_classic that will come to mind later

This will create a period of trouble for everyone using a Git checkout of Boost, even if we do everything perfectly and make no mistakes, because momentarily there will either be a header conflict, or Classic headers won't be present, depending on the order we do some of the steps above. So it would be better to give people an advance warning on the list.

(And then we need to repeat the procedure for the master branch as well, with the removal and the addition again synchronized.)

It's not a trivial procedure and there will probably be other issues we'll have to fix after it. The most important thing is to prepare the list of header files to be moved beforehand, and check it several times. :-) After that it might be easiest if I do all of the above myself, because coordination will be an issue otherwise.

[djowel]

It's not a trivial procedure and there will probably be other issues we'll have to fix after it. The most important thing is to prepare the list of header files to be moved beforehand, and check it several times. :-) After that it might be easiest if I do all of the above myself, because coordination will be an issue otherwise.

Sounds good, Peter! Thank you very much for your time and effort, as always! Fortunately, I've prepared for this eventuality for a long time now. The folder structure is already modular. Spirit "classic" is self-contained.

[djowel]

I agree that this discussion about deprecation should probably be done in the Boost list. The list is so alien to me now, but I'll chime in if there's some discussion. I do see the reason behind @pdimov 's concerns, though. And actually, maintaining classic isn't that much of a burden, compared to Qi/Karma/X3 —as long it is frozen, with no new changes.

That said, I’m not sure if there’s any difference between removing a library and deprecating one, in terms of their impact on dependent libraries. Also, I don’t believe a lifetime commitment is required for a Boost library.

Meanwhile, let's give you access to the repos.

[saki7]

Everyone interested on this matter should comment on this thread. I'm going to pin this discussion.

[saki7]

@djowel @pdimov May I ask you to clarify whether there are any existing deployments of battle-tested parsers built on Classic? This information is important for a broader investigation aimed at understanding the real-world impact—not just hypothetical concerns.

I’m looking for examples of production-level applications used by real companies, not abandoned frameworks with thousands of commits that have sat unmaintained on GitHub for years, as those likely aren’t being used to solve real-world problems.

[djowel]

The actual site seems to be online but gives empty results. I had to use the wayback machine back to 2010. Anyway, here:

https://web.archive.org/web/20100516183106/http://boost-spirit.com/home/info/who-is-using-spirit/

[saki7]

FYI: As the new maintainer, I’ve addressed all open PRs, except for the hard ones that require input from other maintainers ☺️
I’ll continue reviewing the remaining issues.

[djowel]

FYI: As the new maintainer, I’ve addressed all open PRs, except for the hard ones that require input from other maintainers ☺️ I’ll continue reviewing the remaining issues.

Perfect! I welcome you with full embrace, @saki7. Thank you so much for all your help, current and future! I'll be here, constantly, in an advisory capacity, and when things get a bit too challenging with the code, I can also still find some time to do some sleuthing and coding.

[saki7]

I just finished reviewing all open issues. I've triaged and set labels on all existing issues, so enjoy the colorful board :)

[saki7]

For reference, there are 2 issues that might affect the transition of Classic.
See: #805

[djowel]

I've been following the flurry of messages from github. I am pleased and I applaud your dedication! Nice!

[saki7]

Sorry for the noise, but just a heads-up about upcoming changes: I'll be adding labels to all existing PRs, including the closed/merged ones, from now on. I've already done this for regular issues, and I've decided it's necessary for PRs as well. Labels are important for making search work well.