Still new to CrateDB. Have some basic quesions

CrateDB Configuration
1

Hello,

Still very new to CrateDB and have a few basic questions.

I have a CrateDB cluster(version 4.6.6) running using the Prometheus adapter, thanks again to you guys.

Now I would like to understand how to do the following:

  1. Identify the default location of the cratedb files. According to the /etc/crate/crate.yml config file, I can set a path(#path.data: /path/to/data1,/path/to/data2), but it’s not clear where the data is currently stored – as in the exact path.

    • Goal: I want to move the directory off of the “/” file system and store it on a separate disk on the system or an NFS mount point.

  2. I would like to query a database table and do something like: “select * from crate-main-table” to confirm that the Prometheus scraped data is actually being stored in the crate database.

Thanks again!!

2

Hello Brian,

I also started to evaluate CrateDB for one of our projects recently. There’s one issue regarding storage path, maybe yours is similar to mine. Did you install from the official CrateDB APT repository?

If that is the case, what’s described at I'm trying to set up a 3-node CrateDB cluster and now I can't even connect with crash - #2 by proddata might apply to your setup, too:

You installed CrateDB using the apt packages, correct? If you do that, CrateDB starts automatically ins single-node mode which also prevents adding the nodes to any sort of cluster. This is intentional in a way, that apt packages should start without any additional confiuguration.

There are basically two ways to fix this

using the crate-node tool

or (easier)

deleting the cluster state by …
    service crate stop
    rm -rf /usr/share/crate/data
    service crate start

As far as I understand, if you install from CrateDB APT package repository, the default path for CrateDB storage is /usr/share/crate/data.

If you have another disk, and if your system is a GNU/Linux distribution, and if you already mounted your disk, e.g. /path/to/my/disk/ then you can set the path.data: parameter in /etc/crate/crate.yml config file to the path (wherever you mounted your disk, e.g. /path/to/my/disk/) and restart the crate service.

I did something similar, and then observed that CrateDB started to write to that location, e.g. /path/to/my/disk/ (another disk, dedicated to data storage).

3

Hello @Emre_Sevinc,

Thanks for the suggestion.

I actually did not do a complete install, but instead downloaded the binary for the Prometheus adapter.

I went ahead and tried what you did and the files seem to now be writing to the “path.data” location:

As you can see below, the files are being written to the new location:

/<new-disk-path>/nodes/0/_state/
total 32
drwxr-xr-x. 2 crate crate 4096 Jan 16 12:28 .
drwxr-xr-x. 3 crate crate 4096 Jan 16 12:28 ..
-rw-r--r--. 1 crate crate  278 Jan 16 12:28 _3.cfe
-rw-r--r--. 1 crate crate 1461 Jan 16 12:28 _3.cfs
-rw-r--r--. 1 crate crate  382 Jan 16 12:28 _3.si
-rw-r--r--. 1 crate crate  109 Jan 16 12:28 manifest-0.st
-rw-r--r--. 1 crate crate   89 Jan 16 12:28 node-0.st
-rw-r--r--. 1 crate crate  245 Jan 16 12:28 segments_8
-rw-r--r--. 1 crate crate    0 Jan 16 12:28 write.lock

I would still like to know the following:

  1. How to run a “select * from crate” to confirm the data is being written

  2. It would also be good to know where the the cratedb prometheus adapter actually places the data by default.

4

Hello @BrianS ,

I’m glad at least now you can successfully modify path.data and make CrateDB write data files to wherever you want based on that.

I’m afraid I don’t have any experience with Prometheus ↔ CrateDB integration, maybe CrateDB tech support team can help further with that.

As for your following question:

How to run a “select * from crate” to confirm the data is being written

I generally check the existing tables as well as newly created ones, and run SELECT queries by using one of the following methods:

1 Like
5

Hi @BrianS,

If you followed the CrateDB Prometheus Adapter README you should have created a metrics table. You should find your records in this table:

SELECT * FROM metrics ORDER BY timestamp DESC LIMIT 100;

You can access your cluster and execute SQL queries with one of the options @Emre_Sevinc suggested.

Running SQL query SHOW TABLES; will return all table names in your cluster.

6

Hello @Emre_Sevinc,

Thanks for pointing me to the documentation. It was very helpful with navigating the CrateDB options.

I was also able to setup the crash tool CLI , list and describe the table in the database.

I appreciate all of the help everyone is providing thus far.

Thank you!!

4 Likes