Eliminate Help Indices

[merks]

It's a bad idea to build a help index these days. Every project that builds one and includes it in their bundle, provides one that cannot be consumed. Some projects have stopped building one, or don't include the index in the bundle, but nevertheless declare the index in the plugin.xml.

Opening Help Contents and doing a search is really not pretty:

---

I'm not sure how folks are building their index. Maybe with this.

https://tycho.eclipseprojects.io/doc/main/tycho-extras/tycho-document-bundle-plugin/build-help-index-mojo.html

EMF was using Ant and then removed it from the build:

eclipse-emf/org.eclipse.emf@8cabdb5

But I forget to remove the declared index from the plugin.xml, which I fixed during this release cycle

eclipse-emf/org.eclipse.emf@12d1a85#diff-630dec9eedf748888864d0a402774370446e8510cbcb0716a1f83c0b0da2e8d3

Please stop building an index and please remove the <index path="index"/> from every plugin.xml.

---

Not a single bundle indexes with lucene 10.3.2 so not a single index is unsable. Some bundles declare an index but the folder is not present. Some declare an index, the folder is present, but there is no index information in that folder. Some source bundles even include a plugin.xml with an index which is completely bogus. Here is a report.

Bundle Symbolic Name	Bundle Version	Lucene Version	Analyzer Version
org.eclipse.cdt.autotools.docs	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.doc.isv	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.doc.user	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.meson.docs	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.dltk.tcl.doc.user	6.4.2.202509021500	8.0.0.v20190404-1858	4.2.800.v20191210-0610
org.eclipse.emf.cdo.doc	4.2.17.v20251224-1240	index
org.eclipse.emf.mwe.doc	1.19.0.v20260221-0523	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.emf.transaction.doc	1.5.0.202510231413	index
org.eclipse.emf.workspace.doc	1.4.0.202510231413	index
org.eclipse.jst.ejb.doc.user	1.1.301.v201903222024	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.j2ee.doc.user	1.1.400.v202101072335	1.9.1.v200706111724	3.3.1.v20070813_33x
org.eclipse.jst.server.ui.doc.user	1.0.600.v201901310132	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.jst.ws.axis.ui.doc.user.source	1.1.200.v201903222115	indexed_dependencies
org.eclipse.jst.ws.axis.ui.doc.user	1.1.200.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.ws.axis2.ui.doc.user	1.0.200.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.ws.consumption.ui.doc.user.source	1.0.700.v201903222115	indexed_dependencies
org.eclipse.jst.ws.consumption.ui.doc.user	1.0.700.v201903222115	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.jst.ws.doc.user.source	1.0.700.v201903222115	indexed_dependencies
org.eclipse.jst.ws.doc.user	1.0.700.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.net4j.db.doc	4.3.7.v20251224-1240	index
org.eclipse.net4j.doc	4.2.13.v20251224-1240	index
org.eclipse.net4j.util.doc	4.2.15.v20251224-1240	index
org.eclipse.ocl.doc	3.23.0.v20260217-0639	index
org.eclipse.php.help	8.4.0.202512011310	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.qvtd.doc	0.34.0.v20260217-0910	index
org.eclipse.tracecompass.analysis.profiling.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.doc.dev	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.gdbtrace.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.tmf.pcap.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.uml2.doc	5.6.0.v20260216-0916	index
org.eclipse.wst.command.env.doc.user.source	1.5.400.v201903222115	indexed_dependencies
org.eclipse.wst.command.env.doc.user	1.5.400.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.dtdeditor.doc.user	1.1.0.v201903222120	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.wst.jsdt.doc	2.2.0.v202303191832	9.7.0.v20230703-0758	4.4.200.v20230929-1810
org.eclipse.wst.server.ui.doc.user	1.1.600.v201901310132	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.wst.sse.doc.user	1.2.0.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.webtools.doc.user	1.0.500.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.wsdl.ui.doc.user.source	1.0.850.v201903222115	indexed_dependencies
org.eclipse.wst.wsdl.ui.doc.user	1.0.850.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.wsi.ui.doc.user.source	1.0.750.v201903222115	indexed_dependencies
org.eclipse.wst.wsi.ui.doc.user	1.0.750.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.xmleditor.doc.user	1.1.0.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.xsdeditor.doc.user	1.0.800.v201903222120	1.9.1.v20080530-1600	3.3.101.M20080728_34x

The report generator.

package org.eclipse.oomph.toc;

import java.io.IOException;
import java.net.URI;
import java.nio.file.FileSystems;
import java.nio.file.FileVisitResult;
import java.nio.file.Files;
import java.nio.file.NoSuchFileException;
import java.nio.file.Path;
import java.nio.file.SimpleFileVisitor;
import java.nio.file.attribute.BasicFileAttributes;
import java.util.Map;
import java.util.Properties;
import java.util.regex.Pattern;

public class Analyzer {

//	<extension point="org.eclipse.help.toc">
//		<toc file="toc.xml" primary="true"/>
//		<index path="index"/>
//	</extension>

	private static final Pattern HELP_TOC_PATTERN = Pattern
			.compile("<extension.*?point=\"org.eclipse.help.toc\".*?>(.*?)</extension>", Pattern.DOTALL);

	private static final Pattern TOC_INDEX_PATTERN = Pattern.compile("<index.*?path.*?\"([^\"]+)\"", Pattern.DOTALL);

	private static final Pattern BSN_PATTERN = Pattern.compile("(.*)_(.*)\\.jar");

	private static final Pattern ANALYZER_PATTERN = Pattern.compile("org.eclipse.help.base#(.*)\\?.*");

	public static void main(String[] args) throws Exception {

		var path = Path.of("D:/Users/merks/staging-2026-03/plugins");

		System.out.println("| Bundle Symbolic Name | Bundle Version | Lucene Version | Analyzer Version |");
		System.out.println("|---|---|---|---|");

		Files.walkFileTree(path, new SimpleFileVisitor<Path>() {
			@Override
			public FileVisitResult visitFile(Path file, BasicFileAttributes attributes) throws IOException {
				if (file.toString().endsWith(".jar")) {
					try (var fileSystem = FileSystems.newFileSystem(URI.create("jar:" + file.toUri()), Map.of())) {
						var pluginXML = fileSystem.getPath("plugin.xml");
						if (Files.isRegularFile(pluginXML)) {
							var content = Files.readString(pluginXML);
							var tocMatcher = HELP_TOC_PATTERN.matcher(content);
							if (tocMatcher.find()) {
								var tocIndexMatcher = TOC_INDEX_PATTERN.matcher(tocMatcher.group(1));
								if (tocIndexMatcher.find()) {
									var bsnMatcher = BSN_PATTERN.matcher(file.getFileName().toString());
									if (!bsnMatcher.matches()) {
										throw new IllegalStateException();
									}
									System.out.print("| " + bsnMatcher.group(1) + " | " + bsnMatcher.group(2) + " | ");
									var indexPath = fileSystem.getPath(tocIndexMatcher.group(1));
									if (Files.exists(indexPath)) {
										var indexedDependenciesPath = indexPath.resolve("indexed_dependencies");
										try (var indexedDependencies = Files.newInputStream(indexedDependenciesPath)) {
											var indexedDependenciesProperties = new Properties();
											indexedDependenciesProperties.load(indexedDependencies);
											var lucene = indexedDependenciesProperties.get("lucene");
											var analyzer = indexedDependenciesProperties.get("analyzer");
											var analzyerMatcher = ANALYZER_PATTERN.matcher(analyzer.toString());
											if (!analzyerMatcher.matches()) {
												throw new IllegalStateException();
											}
											System.out.print(lucene + " | " + analzyerMatcher.group(1));
										} catch (NoSuchFileException ex) {
											System.out.print(" ~~indexed_dependencies~~ | ");
										}
									} else {
										System.out.print(" ~~" + tocIndexMatcher.group(1) + "~~ | ");
									}
									System.out.println(" |");
								}
							}
						}
					}
				}
				return FileVisitResult.CONTINUE;
			}
		});
	}
}

[ewillink]

OK. There's a regression problem, but given that we were all encouraged (albeit by mindless cloning) to build a help index, there should be nothing wrong with what we are still doing. The regression is the (? one) bad consumer not the (? 25) traditional producers.

[merks]

I never said there was a regression problem. An older Lucene index version will not be consumed by a newer Lucene installed bundle:

https://github.com/eclipse-platform/eclipse.platform/blob/3750de9a37e13c78ed5cc9de26f0e23d72152cbf/ua/org.eclipse.help.base/src/org/eclipse/help/internal/search/SearchIndex.java#L770-L773

Because we update Lucene whenever there is an update available, such a mismatch happens a lot. Moreover if you are building the index with Tycho it's probably almost always an older index version than what's installed in the latest IDE. In fact there is another Lucene update pending for the start of 2026-06 cycle which pretty much guarantees that to be the case.

No matter what, the index will tend to grow stale over time, the index takes space, and I don't thing modern machines need an index cache in order to make help searching faster.

---

I'm not even sure why you are asking given ocl, qvtd, and uml2 declare an index that isn't present in the bundle, e.g.,

So simply remove it from the plugin.xml resources and call it done.

---

Only cdt and php have an index that is marginally recent. Everyone else is doing something quite useless and doing it poorly. Stop doing it at all, save build time, and save disk space, and save download time.

[ewillink]

Seems like

• 1. there should be a Manifest builder warning that "index" is obsolete
• 2. there should be a Manifest builder warning that "index" points at a missing file
• 3. there should be a org.eclipse.help.doc reader upgrade to ignore "index"
• 1. gives relengs an opportunity to improve
• 2. corrects an actual error
• 3. solves your ugly screen

[merks]

I don’t have so much hope for all the things that should be. What I do myself will be.

In this case I’d settle for participants reading this issue and doing what I ask, and even for that I’m only marginally hopeful because it’s so much less work to read nothing and do nothing. We are all very busy…

[estepper]

FWIW., I've removed that line in my four doc plugins and regenerated them. With no diff resulting.

[merks]

FWIW., I've removed that line in my four doc plugins and regenerated them. With no diff resulting.

The only expected difference would be the plugin.xml itself.

[merks]

I'll maintain a updated report here:

Bundle Symbolic Name	Bundle Version	Lucene Version	Analyzer Version
org.eclipse.cdt.autotools.docs	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.doc.isv	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.doc.user	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.cdt.meson.docs	12.4.0.202512101642	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.dltk.tcl.doc.user	6.4.2.202509021500	8.0.0.v20190404-1858	4.2.800.v20191210-0610
org.eclipse.emf.mwe.doc	1.19.0.v20260221-0523	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.emf.transaction.doc	1.5.0.202510231413	index
org.eclipse.emf.workspace.doc	1.4.0.202510231413	index
org.eclipse.jst.ejb.doc.user	1.1.301.v201903222024	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.j2ee.doc.user	1.1.400.v202101072335	1.9.1.v200706111724	3.3.1.v20070813_33x
org.eclipse.jst.server.ui.doc.user	1.0.600.v201901310132	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.jst.ws.axis.ui.doc.user.source	1.1.200.v201903222115	indexed_dependencies
org.eclipse.jst.ws.axis.ui.doc.user	1.1.200.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.ws.axis2.ui.doc.user	1.0.200.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.jst.ws.consumption.ui.doc.user.source	1.0.700.v201903222115	indexed_dependencies
org.eclipse.jst.ws.consumption.ui.doc.user	1.0.700.v201903222115	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.jst.ws.doc.user.source	1.0.700.v201903222115	indexed_dependencies
org.eclipse.jst.ws.doc.user	1.0.700.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.ocl.doc	3.23.0.v20260217-0639	index
org.eclipse.php.help	8.4.0.202512011310	10.2.2.v20250621-0700	4.5.200.v20250905-0730
org.eclipse.qvtd.doc	0.34.0.v20260217-0910	index
org.eclipse.tracecompass.analysis.profiling.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.doc.dev	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.gdbtrace.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.tracecompass.tmf.pcap.doc.user	11.2.0.202512122003	8.4.1.v20200122-1459	4.3.600.v20220308-0310
org.eclipse.uml2.doc	5.6.0.v20260216-0916	index
org.eclipse.wst.command.env.doc.user.source	1.5.400.v201903222115	indexed_dependencies
org.eclipse.wst.command.env.doc.user	1.5.400.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.dtdeditor.doc.user	1.1.0.v201903222120	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.wst.jsdt.doc	2.2.0.v202303191832	9.7.0.v20230703-0758	4.4.200.v20230929-1810
org.eclipse.wst.server.ui.doc.user	1.1.600.v201901310132	1.9.1.v20080530-1600	3.3.101.M20080728_34x
org.eclipse.wst.sse.doc.user	1.2.0.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.webtools.doc.user	1.0.500.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.wsdl.ui.doc.user.source	1.0.850.v201903222115	indexed_dependencies
org.eclipse.wst.wsdl.ui.doc.user	1.0.850.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.wsi.ui.doc.user.source	1.0.750.v201903222115	indexed_dependencies
org.eclipse.wst.wsi.ui.doc.user	1.0.750.v201903222115	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.xmleditor.doc.user	1.1.0.v201903222120	1.9.1.v200803061811	3.3.100.v20080425
org.eclipse.wst.xsdeditor.doc.user	1.0.800.v201903222120	1.9.1.v20080530-1600	3.3.101.M20080728_34x

[ewillink]

ocl, qvtd, uml2 elimination of index pushed to master for next release.