Revist the content hosted at https://download.eclipse.org/oomph/www/

[merks]

Background

The content located at

• https://download.eclipse.org/oomph/www/

is also served from

• https://www.eclipse.org/setups/

That content is composed from a number sources and parts are copied to the download server via this job, which documents the source and target locations:

• https://ci.eclipse.org/oomph/job/publish-website/

setups.zip

A key resource also maintained at this location is:

• https://download.eclipse.org/oomph/www/setups.zip

which is also served from

• https://www.eclipse.org/setups/setups.zip

That resource is directly maintain by the following job and changes relatively frequently:

https://ci.eclipse.org/oomph/job/setup-archiver/

It is essential that this resource is available via a reliable server with high uptime that is independent of the download server which is relatively unreliable and is prone to DOS-types of downtime. The setups.zip is accessed via a HEAD request by every Eclipse IDE installation on startup.

Maintenance

The Foundation is revisiting its hosting infrastructure, including that for https://www.eclipse.org/setups/

• https://gitlab.eclipse.org/eclipsefdn/infrazilla/-/issues/703

A primary goal of the activity is to eliminate PHP. As such, ideally all of these resources could be replaced with a more modern, secure alternative:

The problematic ones are the ones used from these javascript pages:

So specifically these:

• https://github.com/eclipse-oomph/oomph/blob/master/releng/org.eclipse.oomph.releng/www/installer/fetch.php
• https://github.com/eclipse-oomph/oomph/blob/master/releng/org.eclipse.oomph.releng/www/marketplace/fetch.php

Project-specific Setup Instructions

Pages like this:

• https://github.com/eclipse-oomph/oomph/blob/master/CONTRIBUTING.md#creating-an-eclipse-development-environment

have a link like this

• https://www.eclipse.org/setups/installer/?url=https://raw.githubusercontent.com/eclipse-oomph/oomph/master/setups/configurations/OomphConfiguration.setup&show=true

The setups.zip mentioned above is unzipped to this location:

• https://download.eclipse.org/justj/?file=oomph/www/setups

The www/installer/fetch.php script fetches the content from that subfolder, e.g.,

• https://download.eclipse.org/oomph/www/setups/https/raw.githubusercontent.com/eclipse-oomph/oomph/master/setups/configurations/OomphConfiguration.setup

It uses that to display the information specific to that setup configuration.

Marketplace Health Monitor

A marketplace listing like this:

https://marketplace.eclipse.org/content/github-copilot

has a link to it's tested health status:

E.g.,

https://www.eclipse.org/setups/marketplace/?id=6600208

The www/marketplace/fetch.php script fetches the generated report for that id.

The report information is stored in this very large file:

https://download.eclipse.org/oomph/www/marketplace-listings/marketplace.eclipse.org.setup

The script extracts the information for the specific ID from this large file to avoid transferring the entire file to the client side.

The marketplace.eclipse.org.setup report is generated by this job:

• https://ci.eclipse.org/oomph/job/marketplace-analyzer/

Goal

Replace all *.php content with *.html content. Most of the *.php can be converted to HTML with javascript relatively easily.

The technical challenges here is replacing the two fetch.php scripts because they used in *.html pages.

Installer

It seems feasible in the script to map the URL to an api end point, e.g., to map,

• https://raw.githubusercontent.com/eclipse-oomph/oomph/master/setups/configurations/OomphConfiguration.setup

to

• https://api.github.com/repos/eclipse-oomph/oomph/contents/setups/configurations/OomphConfiguration.setup?ref=master

We'd need to handle gitlab as well...

Marketplace

We could explode the large marketplace.eclipse.org.setup into separate resources that can be fetched directly.

Specifically this location:

oomph/releng/org.eclipse.oomph.releng/www/marketplace/index.html

Lines 452 to 455 in c7e4143

	function loadSingleMarketplaceListings(style, url, id, nodeID) {
	var request = new XMLHttpRequest();
	var fetchURL = "fetch.php?id=" + (id != null ? id : nodeID);
	request.open('GET', fetchURL, true);

could be more like this location

oomph/releng/org.eclipse.oomph.releng/www/marketplace/index.html

Lines 473 to 476 in c7e4143

	function loadMarketplaceListings(style, url) {
	var request = new XMLHttpRequest();
	var requestURL = '../marketplace-listings/marketplace.eclipse.org.setup';
	request.open('GET', requestURL, true);

as long as the javascript can locate the resource from the ID; this is complicated by using the id or the nodeID.

Note that it's a lot of resources, e.g, ~1000 and we might have to store each at two locations, numeric id and symbolic id so then ~2000 resources that are updated daily.