apache / apache/cassandra-ccm

Create DSE without a /conf folder

Open
#749 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.2k
Forks
309
PR merge metrics
No merged PRs in 30d

Description

Trying to build a DSE cluster :
`ccm create --dse dse68 -v 6.8.3 --dse-username=test --dse-password=test -n 3`

Then starting : `ccm start`

```
Traceback (most recent call last):
File "/opt/homebrew/bin/ccm", line 112, in
cmd.run()
File "/opt/homebrew/Cellar/ccm/3.1.5_3/libexec/lib/python3.11/site-packages/ccmlib/cmds/cluster_cmds.py", line 503, in run
if self.cluster.start(no_wait=self.options.no_wait,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/ccm/3.1.5_3/libexec/lib/python3.11/site-packages/ccmlib/dse_cluster.py", line 78, in start
started = super(DseCluster, self).start(no_wait, verbose, wait_for_binary_proto, wait_other_notice, jvm_args, profile_options, quiet_start=quiet_start, allow_root=allow_root, timeout=180)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/ccm/3.1.5_3/libexec/lib/python3.11/site-packages/ccmlib/cluster.py", line 416, in start
node.watch_log_for(start_message, timeout=kwargs.get('timeout',60), process=p, verbose=verbose, from_mark=mark)
File "/opt/homebrew/Cellar/ccm/3.1.5_3/libexec/lib/python3.11/site-packages/ccmlib/node.py", line 510, in watch_log_for
raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", time.gmtime()) + " [" + self.name + "] Missing: " + str([e.pattern for e in tofind]) + ":\n" + reads[:50] + ".....\nSee {} for remainder".format(filename))
ccmlib.node.TimeoutError: 16 Mar 2023 08:36:43 [node1] Missing: ['Starting listening for CQL clients']:
INFO [main] 2023-03-16 09:33:44,992 DseModule.ja.....
See system.log for remainder
```

Looking at the logs in `.ccm/dse68/logs/system.log`
```
INFO [main] 2023-03-16 09:33:45,345 DatabaseDescriptor.java:676 - DiskAccessMode is standard, indexAccessMode is standard, commitlogAccessMode is standard
INFO [main] 2023-03-16 09:33:45,346 DatabaseDescriptor.java:1171 - Detected max direct memory 15.756GiB and max heap 504.000MiB
INFO [main] 2023-03-16 09:33:45,347 DatabaseDescriptor.java:1209 - Using file cache of 7.878GiB, 50% of max direct memory
INFO [main] 2023-03-16 09:33:45,347 DatabaseDescriptor.java:1225 - Using direct reads cache of 40.000MiB, refer to the the direct_reads_size_in_mb property
INFO [main] 2023-03-16 09:33:45,348 DatabaseDescriptor.java:711 - Global memtable memory threshold is enabled at 126 MiB
ERROR [main] 2023-03-16 09:33:45,398 DseModule.java:109 - Exiting...
com.google.inject.CreationException: Unable to create injector, see the following errors:

1) An exception was caught and reported. Message: metadata directory '/var/lib/cassandra/metadata' or, if it does not already exist, an existing parent directory of it, is not readable and writable for the DSE. Check file system and configuration.
at com.datastax.bdp.DseModule.configure(Unknown Source)

1 error
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:466)
at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)
at com.google.inject.Guice.createInjector(Guice.java:96)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at com.datastax.bdp.ioc.DseInjector.get(DseInjector.java:36)
at com.datastax.bdp.DseModule.main(DseModule.java:103)
Caused by: org.apache.cassandra.exceptions.ConfigurationException: metadata directory '/var/lib/cassandra/metadata' or, if it does not already exist, an existing parent directory of it, is not readable and writable for the DSE. Check file system and configuration.
at org.apache.cassandra.config.DatabaseDescriptor.resolveAndCheckDirectory(DatabaseDescriptor.java:1267)
at org.apache.cassandra.config.DatabaseDescriptor.applySimpleConfig(DatabaseDescriptor.java:730)
at org.apache.cassandra.config.DatabaseDescriptor.applyAll(DatabaseDescriptor.java:569)
at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:233)
at org.apache.cassandra.config.DatabaseDescriptor.daemonInitialization(DatabaseDescriptor.java:253)
at com.datastax.bdp.DseCoreModule.(DseCoreModule.java:84)
at com.datastax.bdp.DseModule.getRequiredModules(DseModule.java:163)
at com.datastax.bdp.server.AbstractDseModule.configure(AbstractDseModule.java:27)
at com.datastax.bdp.DseModule.configure(DseModule.java:89)
at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements.getElements(Elements.java:110)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
... 5 common frames omitted
```

Everything ccm do must be in the `.ccm` folder.

I wanted to check the conf folder but it's not here:
```
tree -L 2
.
├── cluster.conf
├── node1
│ ├── bin
│ ├── commitlogs
│ ├── data
│ ├── keys
│ ├── logs
│ ├── node.conf
│ ├── resources
│ └── saved_caches
├── node2
│ ├── bin
│ ├── commitlogs
│ ├── data
│ ├── keys
│ ├── logs
│ ├── node.conf
│ ├── resources
│ └── saved_caches
└── node3
├── bin
├── commitlogs
├── data
├── keys
├── logs
├── node.conf
├── resources
└── saved_caches
```

Creating an OSS C* cluster with : `ccm create test -v 4.1.0 -n 3`

The `/conf` folder is here.

```
tree -L 2
.
├── cluster.conf
├── node1
│ ├── bin
│ ├── commitlogs
│ ├── conf
│ ├── data0
│ ├── hints
│ ├── logs
│ ├── node.conf
│ └── saved_caches
├── node2
│ ├── bin
│ ├── commitlogs
│ ├── conf
│ ├── data0
│ ├── hints
│ ├── logs
│ ├── node.conf
│ └── saved_caches
└── node3
├── bin
├── commitlogs
├── conf
├── data0
├── hints
├── logs
├── node.conf
└── saved_caches
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the DSE 6.8.3 setup with `ccm create` and `ccm start`, then inspect `ccmlib/dse_cluster.py`, `cluster.py`, `node.py`, and the generated `system.log`. Compare the DSE node layout with the OSS cluster's `conf` directory and trace the configured metadata path. Done means the reported DSE cluster starts successfully with the expected generated files.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.