Using Packrat in a server environment without touching pre-loaded packages

I'm moving this conversation from the Google discussion group. Thanks to Kevin Ushey for the response and suggestion.

I have a potentially unusual use case. I'd like to use Packrat when running R scripts under Spotfire Statistics Server. Spotfire is an enterprise platform that provides visualisation, analytics, data exploration, BI etc. functions and includes R integration. A Spotfire user can write R scripts and request that they execute on the Statistics Server, configuring data to be passed between Spotfire and the script (input and output R variables).

Spotfire uses some custom packages to handle marshalling data between R and Spotfire, with the packages stored in a special library folder associated with the Statistics Server software. It then adds R wrapper code around the user provided script to call those packages and marshal the data, before executing R to run the script. We have several Statistics servers installed and the requests to run scripts can be load balanced to any of those servers, plus we setup new servers whenever we upgrade Spotfire (and occasionally to add capacity). I'd like to embed a Packrat based script in my 'user provided' script for a few reasons - mostly because I want to develop outside of Spotfire and then push the correct set of packages into a shared place that other users can make use of without having to deal with the complication of finding the right set of packages to copy to our servers, but also because I'd like to get the advantages of Packrat handling updates to packages appropriately as we add servers or upgrade the R version on the servers..

I can get a 'hacky' solution to work by writing a simple script that sources the '.rprofile' file in my Packrat project folder and forces initialisation. The problem is that Packrat unloads the Spotfire custom packages (quite understandable), plus removes the user library locations from the library path - thus I can't simply set the Spotfire packages to be external because Packrat can't find them in the system library. I've then hacked a little further so I capture the .libPaths prior to initialising Packrat, then place them back on the path and manually load the Spotfire packages, but I'm hoping there's a better way to do this! Ideally I'd be able to initialise Packrat and have it install the appropriate packages, but leave the existing libraries loaded. I'd particularly like to be able to do all this without hardcoding any Spotfire package names - I have no control over whether the vendor may choose to add packages (or remove them) in future, so one concern is that my scripts will break given the hardcoded names (I guess I could write something clever to find any packages that are loaded and reload them after Packrat does it's thing, but surely it would be easier to have an option for Packrat to leave things loaded?).

I came up with a second 'hacky' solution where instead of sourcing the '.rprofile' file I load Packrat and run the restore() and on() functions, passing the clean.search.path parameter to stop the packages from unloading. I'm still not happy with this approach - it relies on me having a version of Packrat available to the server rather than being able to bootstrap using something similar to the .rprofile file.

Kevin suggested adding Packrat project options such as:

• 'external.libpaths': a set of library paths to append to the set of Packrat library paths
• 'clean.search.path': whether existing packages on the search path should be unloaded when activating this project

The problem with this is that the options would apply to all users of the project rather than just applying when I use the project on my servers. I'm thinking that what I really want is some way to run the bootstrap script and pass in options - so prior to sourcing '.rprofile' I could perhaps set environment variables to say "don't clean the search path" and "leave external libpaths intact".

Start by reading the bootstrap .rprofile flow and the restore() and on() entry points, focusing on clean.search.path and external library paths. Define how per-invocation options could preserve pre-loaded packages and external library paths without changing project-wide settings; done means the Spotfire packages remain usable while Packrat initializes and restores dependencies.