problem of convergence from recent commit

[vbartolino]

I'm experiencing issues of convergence with the multispecies model across different commits.

I've tested the model rin_ven_her_g3/gadget3/05.03.6.3.
The previous gadget3 commit 8345b51 was able to find a reasonable LL minimum starting from single species parameters estimates, while the recent commit e8072a0 seems to fail. Not sure at which commit the problem started (maybe version 0.13-0?) See below their respective outcomes on the same model

outer mgc:  8.719629e+15 
initial  value 895509877263693.750000 
outer mgc:  8.719629e+15 
outer mgc:  8.491839e+15 
outer mgc:  7.649717e+15 
outer mgc:  7.628535e+15 
outer mgc:  7.622035e+15 
outer mgc:  7.246168e+15 
outer mgc:  5.26795e+15 
outer mgc:  4.501324e+14 
outer mgc:  1803042137 
outer mgc:  1746527887 
iter  10 value 74887782.047020
outer mgc:  1733236940 
outer mgc:  1716323443 
outer mgc:  1711969111 
outer mgc:  1710228078 
outer mgc:  1709085778 
outer mgc:  1709026429 
outer mgc:  1708921708 
outer mgc:  1682268275 
outer mgc:  1029133463 
outer mgc:  1561080298 
iter  20 value 68641973.961886
outer mgc:  1637457001 
outer mgc:  1643235193 
outer mgc:  1662194584 
outer mgc:  1664400608 
outer mgc:  1658923339 
...
iter1660 value 25114.568195
outer mgc:  109.0678 
outer mgc:  101.6734 
outer mgc:  95.79188 
outer mgc:  84.03946 
outer mgc:  68.42705 
outer mgc:  32.61606 
final  value 25114.568170 
converged
outer mgc:  24.9757 

outer mgc:  8.014714e+15 
initial  value 796878214570138.625000 
outer mgc:  8.014714e+15 
outer mgc:  7.533286e+15 
outer mgc:  7.495932e+15 
outer mgc:  7.02632e+15 
outer mgc:  6.968911e+15 
outer mgc:  6.869036e+15 
outer mgc:  6.864157e+15 
outer mgc:  6.945301e+15 
outer mgc:  6.94941e+15 
outer mgc:  6.949413e+15 
iter  10 value 476714231199375.875000
outer mgc:  6.949416e+15 
outer mgc:  6.949418e+15 
outer mgc:  6.949419e+15 
outer mgc:  6.949419e+15 
outer mgc:  6.949419e+15 
outer mgc:  6.949441e+15 
outer mgc:  6.949441e+15 
outer mgc:  6.953943e+15 
outer mgc:  6.953944e+15 
outer mgc:  6.954978e+15 
iter  20 value 476605963963723.750000
outer mgc:  6.954978e+15 
final  value 476605963171236.687500 
converged
outer mgc:  6.954978e+15 

[vbartolino]

not sure to what extent it's useful, but I found out that 03d73f3 was still succeeding while e327402 fails

[lentinj]

It's certainly interesting, although there's no obvious smoking gun between the 2 commit ranges. I'll generate the model with each commit and see what's different.

[lentinj]

If you save the R code generated from each revision of the model, then the difference seems to be this commit 297df53 from #217:

--- 05.03.6.3.03d73f3f.R        2025-07-01 14:49:09.324153574 +0000
+++ 05.03.6.3.e3274027.R        2025-07-01 14:53:32.909245650 +0000
@@ -3763,14 +3763,14 @@
                   hunt.rin__area_idx <- (1L)
                   predator_area <- area
                   total_predsuit <- hunt.rin__totalsuit[hunt.rin__area_idx]
-                  rin_allM_hunt.rin__cons[, rin_allM__area_idx, rin_allM__age_idx] <- rin_allM_hunt.rin__suit[, rin_allM__area_idx, rin_allM__age_idx] * if (cur_year_projection)
+                  rin_allM_hunt.rin__cons[, rin_allM__area_idx, rin_allM__age_idx] <- rin_allM_hunt.rin__suit[, rin_allM__area_idx, rin_allM__age_idx] * (if (cur_year_projection)
                     ((quota_assess_hunt.rin__var[max(((cur_time - 1L + 1L)%/%step_count) + 1L, 1L)]/total_predsuit) * nvl(pt.rin.hunt.rin.quota.step[[paste(cur_step, sep = ".")]], {
                       warning("No value found in g3_param_table rin.hunt.rin.quota.step, ifmissing not specified")
                       NaN
                     }))
                   else ((if (area != 1)
                     0
-                  else intlookup_getdefault(huntrin, (cur_year * 100L + cur_step), 0))/total_predsuit)
+                  else intlookup_getdefault(huntrin, (cur_year * 100L + cur_step), 0))/total_predsuit)) * rin_allM__wgt[, rin_allM__area_idx, rin_allM__age_idx]
                 }
             }
             rin_allM__totalpredate[] <- nonconform_add(rin_allM__totalpredate[], rin_allM_hunt.rin__cons[, , ])

Whilst I still think 297df53 does the right thing, I'll re-check though. Regardless, hunting pressure after the change will be a lot higher, and something else needs to adjust.

It might be interesting to change unit temporarily to "biomass" in ringedseal_g3/gadget3/06.12.7.7.2-setup/setup_fleets.R, to see if that helps. Obviously that's not a fix, but at least helps pinpoint what's going on.

[lentinj]

(also bits of 06.12.7.7.3-setup are missing, so I'm using 06.12.7.7.2-setup, but I'm assuming it doesn't affect the above.

[vbartolino]

(also bits of 06.12.7.7.3-setup are missing, so I'm using 06.12.7.7.2-setup, but I'm assuming it doesn't affect the above.

sorry for that, now it should be there, but the two are nearly identical

[vbartolino]

It might be interesting to change unit temporarily to "biomass" in ringedseal_g3/gadget3/06.12.7.7.2-setup/setup_fleets.R, to see if that helps. Obviously that's not a fix, but at least helps pinpoint what's going on.

I've tried this with ringedseal_g3/gadget3/06.12.7.7.3-setup and the optimiser find a reasonable minimum

[lentinj]

Possibly an obvious question, but are you using the same optimised results from ringedseal_g3 in both cases? Presumably ringedseal_g3 should optimise to a different point pre & post 297df53.

[vbartolino]

Possibly an obvious question, but are you using the same optimised results from ringedseal_g3 in both cases? Presumably ringedseal_g3 should optimise to a different point pre & post 297df53.

ringedseal_g3 was not re-optimised, so I'm using the same optimised results across these comparisons. I'm not sure which gadget3 commit was used initially for that estimation. If useful, I can test estimates of ringedseal_g3 pre & post 297df53

[lentinj]

If useful, I can test estimates of ringedseal_g3 pre & post 297df53

Yeah, I think that's worthwhile. At very least if ringedseal_g3 is also problematic then that's quicker to re-run :)

[vbartolino]

If useful, I can test estimates of ringedseal_g3 pre & post 297df53

Yeah, I think that's worthwhile. At very least if ringedseal_g3 is also problematic then that's quicker to re-run :)

they are found here
https://github.com/vbartolino/g3_GoB/tree/main/ringedseal_g3/gadget3/models/06.12.7.7.3-baseline_a7590fcaa
https://github.com/vbartolino/g3_GoB/tree/main/ringedseal_g3/gadget3/models/06.12.7.7.3-baseline_fedfd771

the params and weights estimated are different but I would say reasonably close...

is the gadget3 version (I mean the id of the commit) saved somewhere with the r and tmb models (and why not copied also in the fit obj)? I feel it would be useful

[lentinj]

is the gadget3 version (I mean the id of the commit) saved somewhere with the r and tmb models (and why not copied also in the fit obj)? I feel it would be useful

The gadget3 & TMB version is part of the filename generated when saving to the work dir, e.g. g30_13_0_999_tmb1_9_14_e4a7a9132a215c5391e7ce16210e6a64a00936f4, but not into the object itself. That's easy enough to add though.

The git commit SHA is a hard question though, it's not something that R itself records as part of the install, so a package has no awareness of what version it is, beyond "0.13.0.999" (999 for an unreleased version, itself an unofficial convention). Maybe the problem here is I'm not making enough releases, but it's hard to know where the line is.

Using renv to record the snapshot of all packages that go into a model would help solve this more generally, but not really something gadget3 can help you with.

[lentinj]

the params and weights estimated are different but I would say reasonably close...

Yeah, cdist_sumofsquares_mat.rin_weight and adist_surveyindices_log_si.rin_weight are fairly different but it's certainly not a substantial change. And so is their nll. At least the total amount caught by hunt is substantially higher, as you'd expect.

[vbartolino]

The git commit SHA is a hard question though, it's not something that R itself records as part of the install, so a package has no awareness of what version it is

can't it be found somewhere here?
help(package="gadget3", help_type="text")$info[[1]]

[lentinj]

Depends how you install it. If you use remotes::install_github(), then it will note packageDescription("gadget3")$RemoteSha, if you use remotes::install_local() (because you're developing gadget3), then you don't get anything.

(why it doesn't ask the git repo in the given directory I dunno. It would be nice if it did, but I'm guessing this is more for their dependency-version-detection than our benefit)

OTOH, I originally looked into this for the filename caches, where a familiar string is more useful. Maybe here working some of the time it could be useful..