Summary of pull requests and commits.

This summary is divided into 3 parts (accorordered according to the main project goals

Overview

The following sections summarize the pull request and commits made for this project. They refer to different packages and repositories:

1. RcppSMC: the main R package with its underlying C++ template library; PRs and commits refer to extensions of this template library
2. SVmodelRcppSMC: an R package that illustrates how a potential user of RcppSMC can develop her own package quickly start implementing new algorithms that make use of the facilities implemented under 1.; taking the SV model as a toy example, ancestral line tracking and the Particle Gibbs algorithm are implemented
3. SVmodelExamples: a demo repository of R scripts that run the implemented example facilities of SVmodelRcppSMC for different parameter setups and algorithmic settings

The final section Miscellaneous summarizes PRs and additions that were made during the project. They do not directly relate to the project goals but rather focus on minor “nice to have” additions such as the improvement of user output readability of existing package examples or updates to the documentation prior to a new release on CRAN that is currently discusses.

RcppSMC

Project Goal 1: tracking of ancestral lines

Add facilities that allow for tracking of ancestral lines

• Link: #56
• No. of commits: squashed into 1
• Total changes:
  • Files changed: 3
  • Additions: +120
  • Deletions: -9
• Summary of all commit messages:

1. Change history-element class:
   1. ancestorIndices: stores resampling indices
   2. SetAIndices: sets ancestorindices of historyelement object; overload history.Set() to also allow setting the ancestor indices
   3. GetAIndices: returns ancestorIndices of historyelement object
2. Add three main functions to sampler.h:
   1. GetAPop: re-orders population elements stored in History according to ancestral lines; add some documentation
   2. GeALineInd: returns a vector of ancestor indices given particle index (of the final iteration)
   3. GetALineSpace: returns ancestral line (i.e. a vector of Space-class objects of length corresponding to the length of the History container) given particle index (of the final iteration)
3. Add helper functions (optional):
   1. GetuRSIndices: returns current resampling indices
   2. GetuRSIndex: returns n’th element of current resampling indices
4. Adjust sampler.h to allow for tracking of ancestral lines:
   1. switch htHistoryMode == HistoryType::AL added to enforce storage of ancestral lines; use “switch”-type cases for setting the historymode type
   2. uRSIndices are set to 0,1,...,N-1 if resampling is not performed; in case of htHistoryMode == HistoryType::AL this sets the ancestors correctly but is redundant if htHistoryMode == HistoryType::RAM
   3. add HistoryType::NONE to switch to avoid compiler warnings
5. Update ChangeLog

Project Goal 2: support for conditional SMC

Conditional SMC

• Link: #50
• No. of commits: 19
• Total changes:
  • Files changed: 9
  • Additions: +824
  • Deletions: -14
• Summary of all commit messages:
  • additions to conditional SMC
    • add specific Initialize functions
    • change the order of conditioning: now, conditioning is performed exactly after initialization or propagation
  • TO-DO (IMPORTANT):
    • conditioning coordinate needs to be re-weighted and the weights need to be re-normalized
    • get rid of, or adjust, facilities for adaptation; adaptive resampling; additional MCMC moves and similar
  • user specified weighting function added to moveset.h
  • sampler.h:
    • possible bugfix in base sampler class: line 515 and 727 have integer in if-condition where bool is required, so change to bool via htHistoryMode == HistoryType::AL
    • fix redundant variable in getALineInd and getALineSpace: long N = GetNumber() is never used
  • moveset.h
    • add whitespace in function argument i.e. “time, value, weight,…” instead of “time,value,weight,…” e.g. in l73, 76 etc.
    • add boilerplate for virtual functions for setting conditional particle coordinate and re-weighting appropriately: pfWeight, DoConditionalMove, *defaultWeight - update constructor for moveset-class accordingly to provide placeholder for weighting function
  • sampler.h: changes to conditionalSampler-class
    • add referenceTrajectoryIndices for storing specifically index assignment of the conditional reference path which is useful for cond. SMC resampling schemes
    • set referenceTrajectoryIndices(T) = referenceTrajectoryIndices(T-1) if no resampling is performed and add a conditionalResampling() function (instead of sampler<Space,Params>::Resample) that does a proper conditional resampling not necessarily permuting the particle set but allowing to draw from the lambda-density (see @adam ’s WP)
    • conditionalResampling() currently only supports multinomial resampling for the trivial lambda()=1/N density case (with other resampling schemes to be adjusted for conditional resampling)
  • move conditionalSampler class to separate header file
  • Add conditional resampling facilities second version.
    • Implements Alg. 3-5 of Appendix C of WP: multinomial, stratified, systematic and residual conditional resampling.
    • Alg. 6, conditional residual resampling yet to come.
  • Adding blueprint/testversion of unfinished implementation of Algorithm 6
  • Improve sampling step from discrete uniform distribution.
    • floor(unif_rand()*N) instead of a call to Rcpp::sample() that produced too much overhead when drawing just one uniform index.
  • WIP: fix some errors and search for bugs in systematic/stratified reampling cases.
  • Fixed errors
    • referenceTrajectoryIndices is never initialized with proper length; now takes length of referenceTrajectory
    • derived conditionalSampler class used Resample instead of conditionalResample
    • for some cases wrong indexing of referenceTrajectoryIndices.at(T) = referenceTrajectoryIndices.at(T - 1); changed to correct version referenceTrajectoryIndices.at(T_ + 1) = referenceTrajectoryIndices.at(T);
    • uRSIndices uninitialized in call to conditionalResample();
    • other algorithmic mistakes for stratified and systematic
  • WIP part searching for bugs:
    • see Rcpp::Rcout statements
  • Change explicit naming in derived class to using where possible.
    • Change name lookup in derived conditionalSampler class from explicit naming to using which improves readability of the code.
  • Finalize Algorithm 6 of the WP.
    • Completes blueprint started in commit 4ec3b5f.
  • Add facilities to fail gracefully for facilities not required for CSMC.
    • Some facilities are not required/implemented in the conditionalSampler derived class e.g. adaptation and MCMC moves.
    • Overload anything that works in the base sampler class but not in the conditional derived class a statement that throws an exception, and make adjustments to member functions and data members used in the derived class. Specifically
      1. conditionalSampler.h
      • Delete redundant data members (e.g. nRepeats)
      • Redundant member functions (e.g. SetAdaptMethods) throw an exception
      • Other member functions using above data members and member functions are adjusted or deleted (which also has the benefit of making the code more readable and understandable in case of conditional SMC)
      • access to member function or member variable of other object (e.g. adaptMethods<Space,Params>;) through a pointer (pAdapt->updateForMCMC()) are deleted
      2. history.h
      • add function overload to Set()-member of the historyelement class that does not use nAccepted or nRepeat since that need not be tracked within a conditional sampler that doesn’t make use of adaptation or MCMC moves
      3. smc-exception.h
      • add new exception (message) types to throw when facilities under 1. are used
  • Add support for printing of conditionalSampler class.
    • ostream<< operator overload for derived class to print:
      • particle value
      • particle log-weight
      • particle weight
    • add a digitsPrint integer so that rounding is performed for digitsPrint number of digits
    • Printing of an instance of a conditionalSampler (within e.g. other packages or .cpp files sourced via Rcpp::source()) is safest for Rcpp::Rcout()
    • Because of the above, a dedicated .print() member may be useful.
    • Also, one should add a pretty printing of ancestral lines (either per default in current facilities other via an additional bool argument to a .print() member)
  • Bugfix conditional resampling, rounding and minor changes.
    • add a MoveReferenceParticle() function to call DoConditionalMove member function of moveset-class similar to MoveParticles() function in sampler.h
    • change to deterministic assignment Kt=0 instead of sampling from lambda for the conditional multinomial ressampling case to save one draw
    • change rounding in operator<< overload when printing conditional class:
    • rounding is still performed for log-weights and normalized weights
    • rounding is removed when printing state values since useful implementation of operator<< overload for Space-class cannot be assumed (or should be well documented so the user implements this herself)
    • fix a bug at the end of conditional residual resampling
    • Update ChangeLog with commit history of this branch. conditionalSampler.h:
      • Improve description of conditionalSampler class.
      • Add comment when fixing reference trajectory index to zero for conditional multinomial resampling
      • Fix error when replication particle values in conditionalResampling():
      • pre-copy pPopulation values before replicatoin (based on ancestor indices) such that previous ancestor value won’t be overwritten in for loop
    • smc-exception.h:
      • remove duplicated lines
      • add unique codes to different exception types
  • ChangeLog:
    • adjust line width in ChangeLog
    • conditionalSampler.h:
    • fix missing break; in switch statements from conditionalResample()
    • fix N-1 vs. N sizes in construction of the tmpIterator
    • add missing argument to arma::linspace

SVmodelRcppSMC

To save space, and since this is an accompanying R package with work preliminary for illustration purposes and not additions to the main RcppSMC library, an overview of commits is linked here..

SVmodelExamples

To save space, and since this is an accompanying repository with work preliminary for demo purposes and not additions to the main RcppSMC library, an overview of commits is linked here.

Miscellaneous

Add user output (convergence diagnostics) for PMMH example

• Link: #50
• No. of commits: 4
• Total changes:
  • Files changed: 5
  • Additions: +216
  • Deletions: -153
• Summary of all commit messages:
  • Additions are:
    • adding two arguments to nonLinPMMH_impl() and nonLinPMMH() to alter user output:
      • boolean verbose: if TRUE, then (additionally to percentage completion) mean parameter estimates, current estimated log-likelihood and log-prior values are reported; if FALSE, only percentage of completion is printed
      • int msg_frq: number of iterations after which percentage completion is printed; defaults to every 100 iterations
    • add a ChangeLog entry
    • change check_prog to integer (with integer equality comparison in l.117)
    • use static_cast() instead of double() for all (both) occasions
    • add NumericVector acceptance_rate to output of nonLinPMMH_impl() to store all acceptance rates

Add missing documentation tononLinPMMH() (or moving documentation to roxygen)

• Link: #52
• No. of commits: 2
• Total changes:
  • Files changed: 3
  • Additions: +11
  • Deletions: -5
• Summary of all commit messages:
  • documentation added to args:
    • verbose
    • msg_freq
    • fixed typo msg_feq -> msg_freq and fequencies -> frequencies

Documentation and miscellaeneous

THIS PR IS WIP

• Link: #61
• No. of commits: 2
• Total changes (so far):
  • Files changed: 9
  • Additions: +56
  • Deletions: -42
• Summary of all commit messages: Update of TODO, README, ChangeLog, standardize copyright headers.
  • TODO:
    • “add extra functionality to the history to store all particles and ancestors”: removed (done)
  • “Update and standardize copyright headers”: removed because done (but may require more updates or different standardization)
  • README: add SVmodelRcppSMC and SVmodelExamples as package and demo repository, respectively, to be listed along the other references in “More Examples”
  • all header files that got committed:
    • standardization: remove unnecessary whitespace and redundant empty comment lines
    • update: include “Ilya Zarubin” to headers that received changes
  • sampler.h: fix documentation error SMC_HISTORY_NONE -> HistoryType::NONE
  • Preliminary updates to NEWS.Rd for release 0.2.4