Provide configuration option to declare CQL scripts that create a "named" Keyspace on startup [DATACASS-723]

John Blum opened DATACASS-723 and commented

The o.s.d.c.core.cql.session.init.SessionFactoryInitializer is not sufficient to replace the o.s.d.c.config.AbstractSessionConfiguration class's now deprecated getStartupScripts() and getShutdownScripts() methods.

As as application developer using Spring Data for Apache Cassandra, if I want to specify a Cassandra Keyspace used by my application, then in my application configuration, I might do, or start with the following, which is very useful...

package ...;

import org.springframework.data.cassandra.config.AbstractCassandraConfiguration;

class MyCassandraApplicationConfiguration extends AbstractCassandraConfiguration {

  @Override
  public String getKeyspaceName() {
    return "MyAppKeyspace";   
  }

  ...
}

However, without also specifying the now deprecated methods, for example...

...

class MyCassandraApplicationConfiguration extends AbstractCassandraConfiguration {

  @Override
  protected List<String> getStartupScripts() {
    return Collections.singletonList("schema.cql");
  }

  @Override
  protected List<String> getShutdownScripts() {
    ...
  }

...
}    

Where schema.cql is defined as...

CREATE KEYSPACE IF NOT EXISTS MyAppKeyspace WITH replication = { 'class':'SimpleStrategy', 'replication_factor':1 };
USE MyAppKeyspace;
CREATE TABLE IF NOT EXISTS customers (id BIGINT PRIMARY KEY, name TEXT);
CREATE INDEX IF NOT EXISTS CustomerNameIdx ON customers(name);
...

Then the application will throw an Exception on startup stating that the named/specified Keyspace (i.e. MyAppKeyspace does not exist!

java.lang.IllegalStateException: Failed to load ApplicationContext
    ....
    ..
    .
Caused by: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'cassandraSessionFactory' defined in example.app.crm.config.CassandraConfiguration: Unsatisfied dependency expressed through method 'cassandraSessionFactory' parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in example.app.crm.config.CassandraConfiguration: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.InvalidKeyspaceException: Invalid keyspace customerservice
	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:798)
	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:539)
    ....
    ..
    .
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cassandraSession' defined in example.app.crm.config.CassandraConfiguration: Invocation of init method failed; nested exception is com.datastax.oss.driver.api.core.InvalidKeyspaceException: Invalid keyspace customerservice
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1796)
    ...
    ..
    .
Caused by: com.datastax.oss.driver.api.core.InvalidKeyspaceException: Invalid keyspace customerservice
	at com.datastax.oss.driver.api.core.InvalidKeyspaceException.copy(InvalidKeyspaceException.java:34)
	at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149)
	at com.datastax.oss.driver.api.core.session.SessionBuilder.build(SessionBuilder.java:501)
	at org.springframework.data.cassandra.config.CqlSessionFactoryBean.buildSession(CqlSessionFactoryBean.java:456)
	at org.springframework.data.cassandra.config.CqlSessionFactoryBean.afterPropertiesSet(CqlSessionFactoryBean.java:427)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1855)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1792)
	... 83 more

The SessionFactoryInitializer (or even the KeyspacePopulator registered on the SessionFactoryFactoryBean provided via extension of the AbstractCassandraConfiguration}) is, or are far too late in the initialization process to "initialize" the "application" (-defined) Keyspace using the Cassandra {{Session provided by the SD Cassandra SessionFactory, which is created by the CqlSessionFactoryBean definition and is based off the "name", application-defined Keyspace anyway.

Essentially the problem can be reproduced by:

1. First declaring the application Keyspace name.

2. The Keyspace name is then configured (i.e. set) on the {{CqlSessionFactoryBean} bean definition declared in the
   AbstractSessionConfiguration class, which the AbstractCassandraConfiguration base class, extended by application code to simplify configuration, extends.

NOTE: Notice that, fortunately, the getStartupScripts() and getShutdownScripts() methods are still used in SD Cassandra despite the deprecation and (e.g. this) comment.

NOTE: Because the SessionFactoryInitializer is too late in the initialization process, it technically breaks the contract stated in the comment of getStartupScripts() and getShutdownScripts() methods. However, the comment(s) are also ambiguous because they do not clarify that the startup and shutdown (CQL) scripts are applied before the application "named" Keyspace is created/initialized, as will be witnessed in #3 following...

3. Therefore, the only opportunity to "create" and "initialize" the application "named" Keyspace is directly after a SD Cassandra framework (internal) Session is opened to the Cassandra "system" Keyspace and subsequently initialized that would then further allow additional Keyspaces to be defined, created and initialized, via CQL scripts applied on startup.

NOTE: Indeed, if we trace through the code, we notice that the keyspaceStartupScripts are passed to the executeSpecsAndScripts(..) method, which ultimately executes the CQL statements. The keyspaceStartupScripts were initialized from the now deprecated AbstractCassandraConfiguration.getStartupScripts() method (also see here).

4. However, the very next thing to happen is that now the application "named" Keyspace is created, which leads to the IllegalStateException shown above.

5. If we tried to follow the logic of using a SessionFactoryInitializer to perform the schema actions above, we'd see that A) the Session connected to the application "named" Keyspace would then be supplied by the CqlSessionFactoryBean bean definition from the AbstractSessionConfiguration class (which again, the user's application would indirectly extend) to the SessionFactoryFactoryBean bean definition declared in the AbstractCassandraConfiguration class (the class our application configuration class extends).

6. It is the SessionFactoryFactoryBean that supplies the SessionFactory that ultimately is post processed by the declared SessionFactoryInitializers in the Spring context.

7. Yet, as stated above, this is too late in the initialization process.

See comments below for possible solutions.

Affects: 3.0 M2 (Neumann)