Skip to content

Add publish command#249

Draft
consideRatio wants to merge 2 commits intosensmetry:mainfrom
consideRatio:publish
Draft

Add publish command#249
consideRatio wants to merge 2 commits intosensmetry:mainfrom
consideRatio:publish

Conversation

@consideRatio
Copy link
Copy Markdown
Collaborator

@consideRatio consideRatio commented Apr 1, 2026

Summary

Adds a new sysand publish command to upload .kpar packages to a Sysand-compatible index, including CLI wiring, core publish logic, docs, and test coverage.

What’s included

New CLI command: sysand publish

  • Usage: sysand publish [OPTIONS] [PATH]
  • If PATH is omitted, resolves to default build output for the current project.
  • New option:
    • --index <URL> to explicitly select publish target index.

Publish target resolution

When --index is not provided, publish target is resolved in this order:

  1. configured index with default = true
  2. first configured index
  3. built-in fallback https://beta.sysand.org

Core publish flow

  • Reads and validates project metadata from the .kpar:
    • publisher present and canonicalizable
    • name canonicalizable
    • version valid SemVer 2.0
  • Builds normalized package identifier (pkg:sysand/<publisher>/<name>@<version>):
    • lowercase normalization
    • spaces normalized to -
    • dots in name preserved
  • Uploads multipart form (purl + file) to /api/v1/upload.
  • Uses bearer-token authentication policy for publish requests (basic auth credentials are ignored for publish).
  • CLI now uses separate auth policies: basic-or-bearer for general commands and bearer-only for publish.
  • Fails early with a clear error if no matching bearer token credentials are configured for the publish URL.
  • Maps server responses into typed publish errors/success outcomes.

Docs and CLI help

  • Added sysand publish command docs.
  • Updated command summary navigation.
  • Updated help text and docs to reflect publish index resolution behavior.

Tests

  • Added/expanded publish integration tests in sysand/tests/cli_publish.rs:
    • missing KPAR
    • explicit missing KPAR
    • network error
    • basic auth credentials are ignored for publish
    • explicit error when no matching bearer token credentials are configured
    • invalid --index
    • explicit path outside project dir
    • metadata validation failures (publisher, name, version)
    • canonicalized purl in upload payload
    • default-index and configured-default-index selection behavior
  • Added unit tests for:
    • publish index URL resolution logic
    • upload URL construction behavior (path append, query/fragment normalization, invalid schemes)

Signed-off-by: Erik Sundell <erik.sundell@sensmetry.com>
@consideRatio consideRatio marked this pull request as ready for review April 1, 2026 13:51
@andrius-puksta-sensmetry
Copy link
Copy Markdown
Collaborator

Uploads multipart form (purl + file) to /api/v1/upload

Why not use something like Cargo publish protocol?

  • Reads and validates project metadata from the .kpar:
    • publisher present and canonicalizable
    • name canonicalizable
    • version valid SemVer 2.0

What about license? Should require it to be an SPDX expression.

/// configured default index URL, otherwise the first configured
/// index URL, or https://beta.sysand.org
#[arg(long, verbatim_doc_comment)]
index: Option<String>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
index: Option<String>,
index: Option<Url>,

#[arg(long, short, default_value_t = false, verbatim_doc_comment)]
allow_path_usage: bool,
},
/// Publish a KPAR to the sysand package index
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Publish a KPAR to the sysand package index
/// Publish a KPAR to a sysand package index

Any index, not necessarily "the" index (beta.sysand.org)

Comment on lines +274 to +275
let basic_or_bearer_auth_policy = Arc::new(auths_builder.build()?);
let bearer_only_auth_policy = Arc::new(publish_auths_builder.build()?);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's avoid doing additional work for every command. Publish is very rare. Better add a conversion method from the former to the latter, and use that for the publish command.

@andrius-puksta-sensmetry
Copy link
Copy Markdown
Collaborator

Uploads multipart form (purl + file) to /api/v1/upload

Why not use something like Cargo publish protocol?

Alaternatively, why not be even simpler: pass the PURL in some header (e.g. X-Sysand-publish-PURL) and the KPAR as the application/zip body?

MissingPublisher,

#[error(
"publisher field `{0}` is invalid for modern project IDs: must be 3-50 characters, use only letters and numbers, may include single spaces or hyphens between words, and must start and end with a letter or number"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"publisher field `{0}` is invalid for modern project IDs: must be 3-50 characters, use only letters and numbers, may include single spaces or hyphens between words, and must start and end with a letter or number"
"publisher field `{0}` is invalid for modern project IDs: must be 3-50 characters, use only ASCII letters and numbers, may include single spaces or hyphens between words, and must start and end with a letter or number"

pub is_new_project: bool,
}

fn build_upload_url(index_url: &str) -> Result<Url, PublishError> {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take a parsed URL.

url: index_url.into(),
reason: "URL cannot be used as a hierarchical base URL".to_string(),
})?;
segments.pop_if_empty();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Require the URL to be HTTP(S), then unwrap() can be used here, and the errors will be better.

Comment on lines +147 to +149
// Keep publish endpoint path stable even if user provided query/fragment in base URL.
upload_url.set_query(None);
upload_url.set_fragment(None);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO it would be better to error out if query/fragment is present, instead of silently dropping them.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or log::warn

let version = &info.version;
if !is_canonicalizable_publisher_field_value(publisher) {
return Err(PublishError::NonCanonicalizablePublisher(
publisher.to_owned().into_boxed_str(),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
publisher.to_owned().into_boxed_str(),
publisher.as_str().into_boxed_str(),

}
if !is_canonicalizable_name_field_value(name) {
return Err(PublishError::NonCanonicalizableName(
name.to_owned().into_boxed_str(),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
name.to_owned().into_boxed_str(),
name.as_str().into_boxed_str(),

));
}
semver::Version::parse(version).map_err(|source| PublishError::InvalidVersion {
version: version.to_owned().into_boxed_str(),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
version: version.to_owned().into_boxed_str(),
version: version.as_str().into_boxed_str(),

let request_builder = move |c: &reqwest_middleware::ClientWithMiddleware| {
let file_part = reqwest::multipart::Part::stream(file_bytes.clone())
.file_name(file_name.clone())
.mime_str("application/octet-stream")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not application/zip?

let file_part = reqwest::multipart::Part::stream(file_bytes.clone())
.file_name(file_name.clone())
.mime_str("application/octet-stream")
.expect("valid mime type");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.expect("valid mime type");
.unwrap();

.text("purl", purl.clone())
.part("file", file_part);

c.post(upload_url.as_str()).multipart(form)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
c.post(upload_url.as_str()).multipart(form)
c.post(upload_url).multipart(form)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this won't work. Then clone the URL to avoid very expensive reparse.

pub fn do_publish_kpar<P: AsRef<Utf8Path>>(
kpar_path: P,
index_url: &str,
auth_policy: Arc<StandardHTTPAuthentication>,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We always use bearer auth and no registry will allow unauthenticated publishing, so use ForceBearerAuth.


pub fn do_publish_kpar<P: AsRef<Utf8Path>>(
kpar_path: P,
index_url: &str,
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take a URL.

Comment on lines +38 to +41
let is_separator = b == b'-' || b == b' ' || (allow_dot && b == b'.');
if !is_separator {
return false;
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This allows non-ASCII to pass through.

})?;

let status = response.status().as_u16();
let body = runtime.block_on(response.text()).unwrap_or_default();
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't eat errors.


#[test]
fn publisher_field_canonicalizability() {
assert!(is_canonicalizable_publisher_field_value("Acme Labs"));
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a case for dot in the middle.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also a case for - (space and punctuation), and one or more cases of non-ASCII.

@@ -0,0 +1,77 @@
# `sysand publish`

Publish a KPAR to the sysand package index.
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Publish a KPAR to the sysand package index.
Publish a KPAR to a sysand package index.


- `version`: must be a valid Semantic Versioning 2.0 version.

- `publisher`: 3-50 characters, letters and numbers only, with optional single
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `publisher`: 3-50 characters, letters and numbers only, with optional single
- `publisher`: 3-50 characters, ASCII letters and numbers only, with optional single

- `publisher`: 3-50 characters, letters and numbers only, with optional single
spaces or hyphens between words, and must start and end with a letter or
number.
- `name`: 3-50 characters, letters and numbers only, with optional single
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `name`: 3-50 characters, letters and numbers only, with optional single
- `name`: 3-50 characters, ASCII letters and numbers only, with optional single

## Options

- `--index <URL>`: URL of the package index to publish to. Defaults to the
configured default index URL, otherwise the first configured index URL, or
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Link to index config

Comment on lines +45 to +55
if let Some(index_url) = config
.indexes
.iter()
.find(|index| index.default.unwrap_or(false))
.map(|index| index.url.as_str())
.or_else(|| config.indexes.first().map(|index| index.url.as_str()))
{
Url::parse(index_url).map_err(|e| anyhow!("invalid index URL in configuration: {e}"))
} else {
Ok(Url::parse(DEFAULT_INDEX_URL).expect("default publish index URL must be valid"))
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid all this parsing, use Url in config struct. If this requires many changes, just add a TODO there.

) -> Result<()> {
let kpar_path = resolve_publish_kpar_path(path, ctx)?;
if !kpar_path.is_file() {
bail!("kpar file not found at `{kpar_path}`, run `sysand build` first");
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
bail!("kpar file not found at `{kpar_path}`, run `sysand build` first");
bail!("KPAR file not found at `{kpar_path}`, run `sysand build` first");

runtime: Arc<tokio::runtime::Runtime>,
) -> Result<()> {
let kpar_path = resolve_publish_kpar_path(path, ctx)?;
if !kpar_path.is_file() {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wrapfs::is_file

{
Url::parse(index_url).map_err(|e| anyhow!("invalid index URL in configuration: {e}"))
} else {
Ok(Url::parse(DEFAULT_INDEX_URL).expect("default publish index URL must be valid"))
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Ok(Url::parse(DEFAULT_INDEX_URL).expect("default publish index URL must be valid"))
Ok(Url::parse(DEFAULT_INDEX_URL).unwrap())

We know it's valid.

let config = Config::default();
let url = resolve_publish_index_url(None, &config).unwrap();

assert_eq!(url.as_str(), format!("{DEFAULT_INDEX_URL}/"));
Copy link
Copy Markdown
Collaborator

@andrius-puksta-sensmetry andrius-puksta-sensmetry Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use concat!:

Suggested change
assert_eq!(url.as_str(), format!("{DEFAULT_INDEX_URL}/"));
assert_eq!(url.as_str(), concat!(DEFAULT_INDEX_URL, '/'));

)
.match_body(Matcher::AllOf(vec![
Matcher::Regex(r#"name="purl""#.to_string()),
Matcher::Regex("pkg:sysand/acme-labs/my\\.project-alpha@1\\.0\\.0".to_string()),
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use raw strings to avoid \\ and similar.

Signed-off-by: Erik Sundell <erik.sundell+2025@sensmetry.com>
@consideRatio consideRatio marked this pull request as draft April 2, 2026 13:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants