Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -259,6 +259,12 @@ Changes:
Cox data before. Both stop once another model has been fitted to the same Cyclops data (fits share its interface;
`survfit()` silently mixed the two fits before) unless that fit used `forceNewObject = TRUE`, and always once
`updateOutcomes()` has changed the data; fit objects keep their `weights`
42. A Laplace fit under `algorithm = "auto"` or `"newton"` that `createControl(newtonMemoryBudget = )` leaves without any
Newton step (its prox-Newton work, 24·|W|² + 12 bytes per entry of the working-set columns, exceeds the budget)
warns, with the memory it needed in `fit$convergence$budgetBytes`. Such fits run coordinate descent, which on
correlated designs can need thousands of cycles: a 500,000-person CCW Cox censoring model (1.2·10^8 entries, 1.4 GB
of prox-Newton work) ran 1,700 cycles in 25 minutes without converging under the 1024 MB default, and converged in
24 iterations and 105 s with `newtonMemoryBudget = 4096`

Cyclops v3.8.0
==============
Expand Down
9 changes: 8 additions & 1 deletion R/ModelFit.R
Original file line number Diff line number Diff line change
Expand Up @@ -474,6 +474,11 @@ fitCyclopsModel <- function(cyclopsData,
is.finite(fit$convergence$gapEstimate)) {
warning("Reached maxIterations; the estimated gap to the optimum is ", signif(fit$convergence$gapEstimate, 3))
}
if (warnings && isTRUE(is.finite(fit$convergence$budgetBytes)) && isTRUE(fit$convergence$newtonIterations == 0)) {
warning("Prox-Newton steps needed about ", signif(fit$convergence$budgetBytes / 2^20, 2), " MB, more than ",
"newtonMemoryBudget = ", if (is.null(control$newtonMemoryBudget)) 1024 else control$newtonMemoryBudget, " MB, so the fit used coordinate ",
"descent, which can be slow or stop short on correlated covariates; raise createControl(newtonMemoryBudget = )")
}

# POOR_BLR_STEP comes from the "gradient" criterion: explicit, the default, or the one a fit that could not take
# Newton steps fell back to under "auto"
Expand Down Expand Up @@ -983,7 +988,9 @@ print.cyclopsFit <- function(x, show.call=TRUE ,...) {
#' conjugate-gradient work. Steps that would exceed it use conjugate gradients or
#' coordinate descent instead. It is a ceiling, not an allocation, and it holds per fit:
#' cross-validation folds or threads that run separate fits may each use it. The default
#' of 1024 allows active sets of up to about 6,700 columns
#' of 1024 allows active sets of up to about 6,700 columns. A Laplace fit that the budget
#' leaves without any Newton step warns, with the memory it needed in
#' \code{fit$convergence$budgetBytes}
#' @param doItAll Currently unused
#' @param syncCV Currently unused
#'
Expand Down
4 changes: 3 additions & 1 deletion man/createControl.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions src/RcppCyclopsInterface.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -905,6 +905,7 @@ static void appendNewtonReports(Rcpp::List& list, bsccs::RcppCcdInterface& rcppI
convergence.push_back(report.faceRounds, "faceRounds");
convergence.push_back(report.dualChecks, "dualChecks");
convergence.push_back(report.eigenFaceRounds, "eigenFaceRounds");
convergence.push_back(report.budgetBytes, "budgetBytes");
list["convergence"] = convergence;
}
}
Expand Down
3 changes: 2 additions & 1 deletion src/cyclops/CyclicCoordinateDescent.h
Original file line number Diff line number Diff line change
Expand Up @@ -297,6 +297,7 @@ class CyclicCoordinateDescent {
int dualChecks = 0; // how many were computed
int eigenFaceRounds = 0; // prox-Newton face rounds solved by an eigendecomposition
std::string ineligible; // why a requested Newton path ran as CCD
double budgetBytes = NAN; // the largest prox-Newton work refused for newtonMemoryBudget (bytes)
};

const ConvergenceReport& getConvergenceReport() const { return convergenceReport; }
Expand Down Expand Up @@ -584,7 +585,7 @@ class CyclicCoordinateDescent {
AlgorithmType algorithmType, int convergenceType);
NewtonOutcome newtonIteration(NewtonContext& context);
NewtonOutcome proxNewtonIteration(NewtonContext& context);
bool proxFitsDense(const std::vector<int>& W) const;
bool proxFitsDense(const std::vector<int>& W); // records a refusal in convergenceReport.budgetBytes
// Memory budget of one fit's Newton work (2.5), in bytes: dense H, its factor and workspace, row-wise copies of
// columns of X (ModeFindingArguments::newtonMemoryBudget, set by update())
double newtonMemoryBudget = 1024.0 * 1024 * 1024;
Expand Down
8 changes: 6 additions & 2 deletions src/cyclops/NewtonSolver.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -934,7 +934,7 @@ double CyclicCoordinateDescent::strongConvexityBound(const NewtonContext& contex
return 0.5 * sum;
}

bool CyclicCoordinateDescent::proxFitsDense(const std::vector<int>& W) const {
bool CyclicCoordinateDescent::proxFitsDense(const std::vector<int>& W) {
const double width = static_cast<double>(W.size());
double entries = 0.0;
// (plr's copy of X_W is by subject, bounded by the nonzeros of X plus K per intercept or basis column: counted
Expand All @@ -946,7 +946,11 @@ bool CyclicCoordinateDescent::proxFitsDense(const std::vector<int>& W) const {
static_cast<double>(hXI.getNumberOfEntries(j));
}
}
return 24.0 * width * width + 12.0 * entries <= newtonMemoryBudget;
const double bytes = 24.0 * width * width + 12.0 * entries;
if (bytes > newtonMemoryBudget && !(bytes <= convergenceReport.budgetBytes)) {
convergenceReport.budgetBytes = bytes; // R warns when this left the fit without Newton steps
}
return bytes <= newtonMemoryBudget;
}

CyclicCoordinateDescent::NewtonOutcome CyclicCoordinateDescent::proxNewtonIteration(NewtonContext& context) {
Expand Down
3 changes: 2 additions & 1 deletion src/cyclops/api/CyclopsApi.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -826,7 +826,8 @@ FitResult Model::Impl::collectResult(double fitSeconds) {
{"rankDeficientFaceRounds", double(report.rankDeficientFaceRounds)},
{"eigenFaceRounds", double(report.eigenFaceRounds)},
{"faceRounds", double(report.faceRounds)},
{"dualChecks", double(report.dualChecks)}};
{"dualChecks", double(report.dualChecks)},
{"budgetBytes", report.budgetBytes}};
result.convergence_text = {{"algorithm", report.algorithm},
{"certificate", report.certificate},
{"solver", report.solver},
Expand Down
10 changes: 8 additions & 2 deletions tests/testthat/test-proxNewton.R
Original file line number Diff line number Diff line change
Expand Up @@ -236,11 +236,17 @@ test_that("newtonMemoryBudget bounds prox-Newton; a tiny budget leaves the fit o
fit <- function(...) fitCyclopsModel(data, prior = createPrior("laplace", variance = 0.1, exclude = 1),
control = createControl(algorithm = "auto", noiseLevel = "silent", ...),
forceNewObject = TRUE)
default <- fit()
tiny <- fit(newtonMemoryBudget = 1e-5) # 10 bytes: below 24 |W|^2 for any active set
default <- expect_no_warning(fit())
# 10 bytes: below 24 |W|^2 for any active set
expect_warning(tiny <- fit(newtonMemoryBudget = 1e-5), "more than newtonMemoryBudget = 1e-05 MB")
expect_equal(default$algorithm, "newton")
expect_gt(default$convergence$newtonIterations, 0)
expect_true(is.na(default$convergence$budgetBytes))
expect_equal(tiny$algorithm, "ccd")
expect_equal(tiny$convergence$newtonIterations, 0)
expect_gt(tiny$convergence$budgetBytes, 10)
expect_no_warning(fitCyclopsModel(data, prior = createPrior("laplace", variance = 0.1, exclude = 1), warnings = FALSE,
control = createControl(algorithm = "auto", noiseLevel = "silent", newtonMemoryBudget = 1e-5),
forceNewObject = TRUE))
expect_equal(coef(tiny), coef(default), tolerance = 1e-6)
})
Loading