CrateDB shows no tables

Shakeel_Ahmed · May 19, 2021, 4:38am

I am new to Cratedb and trying to use this in my project. for testing I have imported IMDB database and everything was going smooth. I was really impressed with select query speed. But suddenly i could not see any data or tables in the admin console and also when I try to connect from CLI.
Possible Reason: Server was rebooted while the service still running. but it should not be so sensitive that I lost the entire tables and shards.
Cratedb version: crate-4.5.0
Cluster setup: Single node
I could able see the data files in the folder structure. can you please help me recover the tables.

proddata · May 19, 2021, 12:17pm

Could you please elaborate what exactly you did.

Are you running CrateDB locally (OS, docker, …)?
which install method did you use?
what folders are you refering to?
what does the startup logs look like?

Shakeel_Ahmed · May 20, 2021, 8:30pm

Thank you for your kind reply… Please find the below reply…

Are you running CrateDB locally (OS, docker, …)?
----yes running in my windows10 laptop with ubuntu subsystem
which install method did you use?
---- crate-4.5.0.tar.gz untar the file and started using it…
what folders are you refering to?
----I am referring to “crate-4.5.0/data/nodes/0/indices” folder

image1164×397 22.4 KB
what does the startup logs look like?
----no error it just starts like a fresh setup and I could not see the created tables.

Startup log-----

[2021-05-21T00:19:54,391][INFO ][o.e.e.NodeEnvironment    ] [Monte Turlon] using [1] data paths, mounts [[/mnt/c (C:\)]], net usable_space [318.4gb], net total_space [475.4gb], types [drvfs]
[2021-05-21T00:19:54,424][INFO ][o.e.e.NodeEnvironment    ] [Monte Turlon] heap size [3.9gb], compressed ordinary object pointers [true]
[2021-05-21T00:19:54,448][INFO ][o.e.n.Node               ] [Monte Turlon] node name [Monte Turlon], node ID [2BItAFyPQwumwyr6X0jrHg], cluster name [crate]
[2021-05-21T00:19:54,541][INFO ][o.e.n.Node               ] [Monte Turlon] version[4.5.0], pid[42], build[fb7a533/2021-03-23T14:12:58Z], OS[Linux/4.4.0-18362-Microsoft/amd64], JVM[AdoptOpenJDK/OpenJDK 64-Bit Server VM/16/16+36]
[2021-05-21T00:19:55,790][INFO ][i.c.plugin               ] [Monte Turlon] plugins loaded: [enterpriseFunctions, jmx-monitoring, lang-js]
[2021-05-21T00:19:58,570][INFO ][o.e.p.PluginsService     ] [Monte Turlon] no modules loaded
[2021-05-21T00:19:58,575][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [crate-azure-discovery]
[2021-05-21T00:19:58,576][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [es-repository-hdfs]
[2021-05-21T00:19:58,576][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [io.crate.plugin.BlobPlugin]
[2021-05-21T00:19:58,577][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [io.crate.plugin.PluginLoaderPlugin]
[2021-05-21T00:19:58,578][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [io.crate.plugin.SrvPlugin]
[2021-05-21T00:19:58,579][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [io.crate.udc.plugin.UDCPlugin]
[2021-05-21T00:19:58,579][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.analysis.common.CommonAnalysisPlugin]
[2021-05-21T00:19:58,580][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.discovery.ec2.Ec2DiscoveryPlugin]
[2021-05-21T00:19:58,584][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.plugin.analysis.AnalysisPhoneticPlugin]
[2021-05-21T00:19:58,585][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.plugin.repository.url.URLRepositoryPlugin]
[2021-05-21T00:19:58,587][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.repositories.azure.AzureRepositoryPlugin]
[2021-05-21T00:19:58,588][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.repositories.s3.S3RepositoryPlugin]
[2021-05-21T00:19:58,589][INFO ][o.e.p.PluginsService     ] [Monte Turlon] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
[2021-05-21T00:20:00,163][INFO ][o.e.d.DiscoveryModule    ] [Monte Turlon] using discovery type [zen] and seed hosts providers [settings]
[2021-05-21T00:20:01,654][INFO ][psql                     ] [Monte Turlon] PSQL SSL support is disabled.
[2021-05-21T00:20:02,379][INFO ][i.c.p.PipelineRegistry   ] [Monte Turlon] HTTP SSL support is disabled.
[2021-05-21T00:20:02,506][INFO ][o.e.n.Node               ] [Monte Turlon] initialized
[2021-05-21T00:20:02,508][INFO ][o.e.n.Node               ] [Monte Turlon] starting ...
[2021-05-21T00:20:02,908][INFO ][psql                     ] [Monte Turlon] publish_address {127.0.0.1:5432}, bound_addresses {[::1]:5432}, {127.0.0.1:5432}
[2021-05-21T00:20:02,952][INFO ][o.e.h.n.Netty4HttpServerTransport] [Monte Turlon] publish_address {127.0.0.1:4200}, bound_addresses {[::1]:4200}, {127.0.0.1:4200}
[2021-05-21T00:20:02,991][INFO ][o.e.t.TransportService   ] [Monte Turlon] publish_address {127.0.0.1:4300}, bound_addresses {[::1]:4300}, {127.0.0.1:4300}
[2021-05-21T00:20:03,122][WARN ][o.e.b.BootstrapChecks    ] [Monte Turlon] I/O exception while trying to read [/proc/sys/vm/max_map_count]
java.nio.file.NoSuchFileException: /proc/sys/vm/max_map_count                                                                                                                   at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~[?:?]
        at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
        at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) ~[?:?]
        at java.nio.file.Files.newByteChannel(Files.java:375) ~[?:?]
        at java.nio.file.Files.newByteChannel(Files.java:426) ~[?:?]
        at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~[?:?]
                                                                                         
at java.nio.file.Files.newInputStream(Files.java:160) ~[?:?]                                                                                                            at java.nio.file.Files.newBufferedReader(Files.java:2916) ~[?:?]
        at java.nio.file.Files.newBufferedReader(Files.java:2948) ~[?:?]
        at org.elasticsearch.bootstrap.BootstrapChecks$MaxMapCountCheck.getBufferedReader(BootstrapChecks.java:448) ~[crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapChecks$MaxMapCountCheck.getMaxMapCount(BootstrapChecks.java:426) [crate-app.jar:4.5.0]                                          at org.elasticsearch.bootstrap.BootstrapChecks$MaxMapCountCheck.getMaxMapCount(BootstrapChecks.java:420) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapChecks$MaxMapCountCheck.check(BootstrapChecks.java:400) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:134) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:91) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:74) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapProxy$1.validateNodeBeforeAcceptingRequests(BootstrapProxy.java:198) [crate-app.jar:4.5.0]
        at org.elasticsearch.node.Node.start(Node.java:716) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapProxy.start(BootstrapProxy.java:205) [crate-app.jar:4.5.0]
        at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:269) [crate-app.jar:4.5.0]
        at io.crate.bootstrap.CrateDB.init(CrateDB.java:158) [crate-app.jar:4.5.0]
        at io.crate.bootstrap.CrateDB.execute(CrateDB.java:138) [crate-app.jar:4.5.0]
        at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:81) [crate-app.jar:4.5.0]
        at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [crate-app.jar:4.5.0]
        at org.elasticsearch.cli.Command.main(Command.java:90) [crate-app.jar:4.5.0]
        at io.crate.bootstrap.CrateDB.main(CrateDB.java:91) [crate-app.jar:4.5.0]
        at io.crate.bootstrap.CrateDB.main(CrateDB.java:84) [crate-app.jar:4.5.0]
[2021-05-21T00:20:03,296][WARN ][o.e.b.BootstrapChecks    ] [Monte Turlon] initial heap size [268435456] not equal to maximum heap size [4202692608]; this can cause resize pauses and prevents mlockall from locking the entire heap. Set the heap size via the CRATE_HEAP_SIZE environment variable. See https://crate.io/docs/crate/howtos/en/latest/performance/memory.html
[2021-05-21T00:20:03,310][INFO ][o.e.c.c.Coordinator      ] [Monte Turlon] cluster UUID [xqhJMLgsQFyheWZo2pKQlQ]
[2021-05-21T00:20:03,342][INFO ][o.e.c.c.ClusterBootstrapService] [Monte Turlon] no discovery configuration found, will perform best-effort cluster bootstrapping after [3s] unless existing master is discovered
[2021-05-21T00:20:03,466][INFO ][o.e.c.s.MasterService    ] [Monte Turlon] elected-as-master ([1] nodes joined)[{Monte Turlon}{2BItAFyPQwumwyr6X0jrHg}{PgLRdPeURwqiG5BnCL_N5A}{127.0.0.1}{127.0.0.1:4300}{http_address=127.0.0.1:4200} elect leader, _BECOME_MASTER_TASK_, _FINISH_ELECTION_], term: 4, version: 13, reason: master node changed {previous [], current [{Monte Turlon}{2BItAFyPQwumwyr6X0jrHg}{PgLRdPeURwqiG5BnCL_N5A}{127.0.0.1}{127.0.0.1:4300}{http_address=127.0.0.1:4200}]}
[2021-05-21T00:20:03,591][INFO ][o.e.c.s.ClusterApplierService] [Monte Turlon] master node changed {previous [], current [{Monte Turlon}{2BItAFyPQwumwyr6X0jrHg}{PgLRdPeURwqiG5BnCL_N5A}{127.0.0.1}{127.0.0.1:4300}{http_address=127.0.0.1:4200}]}, term: 4, version: 13, reason: Publication{term=4, version=13}
[2021-05-21T00:20:03,627][INFO ][o.e.n.Node               ] [Monte Turlon] started
[2021-05-21T00:20:03,760][INFO ][o.e.g.GatewayService     ] [Monte Turlon] recovered [0] indices into cluster_state

proddata · May 21, 2021, 7:59am

Did you change the data path (default: #path.data: data) in the crate.yml?

If there is data/files in that folder and the data would be corrupted, CrateDB normally would not even start.

Shakeel_Ahmed · May 26, 2021, 7:01am

I am sorry for the delay response… Your amazing… i was almost loosing hope on this DB… i tried to configure backup and enabled path.data for backup. after the reboot it was trying to find the files in backup folder… i have changed the config to default and everything back to normal… i could see the entire IMDB records… Amazing… I can confirm CrateDB is crash proof…

jayeff · May 26, 2021, 7:12am

Hi @Shakeel_Ahmed,

great to hear that you got back your data

For backups on filesystem you want to set the path.repo setting. Afterwards you can do a CREATE REPOSITORY to allow snapshotting.

Topic		Replies	Views
Crate db information schema is missing the table we created CrateDB	3	491	April 8, 2022
Missing Schemas CrateDB	1	551	October 9, 2020
ERROR: Garbage collection log directory '/data/log' does not exist or is not accessible CrateDB	11	455	March 14, 2023
Runnig CrateDB from Docker it restarts every few days CrateDB	3	574	April 29, 2022
What can we do when a silo is missing in a table? CrateDB	4	222	August 29, 2023

CrateDB shows no tables

Related Topics