Web3signer 2.0.5 not worked properly (20+ validator)

Before filing a new topic, please provide the following information.

Core DAppNode Packages versions

  • bind.dnp.dappnode.eth: 0.2.9
  • core.dnp.dappnode.eth: 0.2.86
  • dappmanager.dnp.dappnode.eth: 0.2.81, commit: 7bf8e6f5
  • https.dnp.dappnode.eth: 0.2.1
  • ipfs.dnp.dappnode.eth: 0.2.22
  • vpn.dnp.dappnode.eth:
  • wifi.dnp.dappnode.eth: 0.2.9
  • wireguard.dnp.dappnode.eth: 0.1.2

System info

  • dockerComposeVersion: 1.25.0
  • dockerServerVersion: 25.0.0
  • dockerCliVersion: 25.0.0
  • os: debian
  • versionCodename: bullseye
  • architecture: amd64
  • kernel: 5.10.0-27-amd64
  • Disk usage: 27%

2024-01-20 01:00:47.122+00:00 | main | INFO | HikariDataSource | HikariPool-1 - Starting…
2024-01-20 01:00:47.209+00:00 | main | INFO | HikariPool | HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@3e8b3b79
2024-01-20 01:00:47.210+00:00 | main | INFO | HikariDataSource | HikariPool-1 - Start completed.
2024-01-20 01:00:47.366+00:00 | main | INFO | HikariDataSource | HikariPool-2 - Starting…
2024-01-20 01:00:47.384+00:00 | main | INFO | HikariPool | HikariPool-2 - Added connection org.postgresql.jdbc.PgConnection@1e54a6b1
2024-01-20 01:00:47.384+00:00 | main | INFO | HikariDataSource | HikariPool-2 - Start completed.
2024-01-20 01:00:47.587+00:00 | main | INFO | MetricsHttpService | Starting metrics http service on 0.0.0.0:9091
2024-01-20 01:00:47.676+00:00 | vert.x-eventloop-thread-0 | INFO | MetricsHttpService | Metrics service started and listening on 0.0.0.0:9091
2024-01-20 01:00:47.686+00:00 | pool-2-thread-1 | INFO | SignerLoader | Loading signer configuration metadata files from /data/keyfiles
2024-01-20 01:00:47.693+00:00 | pool-2-thread-1 | INFO | SignerLoader | Signer configuration metadata files read in memory 27 in 00:00:00.005
2024-01-20 01:00:47.738+00:00 | pool-2-thread-1 | INFO | SignerLoader | Converting signing metadata to Artifact Signer using parallel streams …
2024-01-20 01:00:47.739+00:00 | ForkJoinPool.commonPool-worker-15 | INFO | SignerLoader | 10 signing metadata processed
2024-01-20 01:00:47.740+00:00 | ForkJoinPool.commonPool-worker-21 | INFO | SignerLoader | 20 signing metadata processed

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread “vertx-blocked-thread-checker”
2024-01-20 01:00:54.620+00:00 | pool-2-thread-1 | INFO | BLS | BLS: loaded BLST library
2024-01-20 01:00:54.624+00:00 | main | ERROR | Runner | Error loading signers
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
at java.util.concurrent.FutureTask.report(Unknown Source) ~[?:?]
at java.util.concurrent.FutureTask.get(Unknown Source) ~[?:?]
at tech.pegasys.web3signer.core.Runner.run(Runner.java:121) [web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.core.Eth2Runner.run(Eth2Runner.java:401) [web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.commandline.subcommands.ModeSubCommand.run(ModeSubCommand.java:33) [web3signer-commandline-24.1.0.jar:24.1.0]
at picocli.CommandLine.executeUserObject(CommandLine.java:1939) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.handle(CommandLine.java:2352) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.handle(CommandLine.java:2314) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.execute(CommandLine.java:2316) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.2.jar:4.6.2]
at tech.pegasys.web3signer.commandline.CommandlineParser.parseCommandLine(CommandlineParser.java:85) [web3signer-commandline-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.Web3SignerApp.executeWithEnvironment(Web3SignerApp.java:50) [web3signer-app-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.Web3SignerApp.main(Web3SignerApp.java:35) [web3signer-app-24.1.0.jar:24.1.0]
Caused by: java.lang.OutOfMemoryError
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source) ~[?:?]
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source) ~[?:?]
at java.lang.reflect.Constructor.newInstanceWithCaller(Unknown Source) ~[?:?]
at java.lang.reflect.Constructor.newInstance(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinTask.getThrowableException(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinTask.reportException(Unknown Source) ~[?:?]
at java.util.concurrent.ForkJoinTask.invoke(Unknown Source) ~[?:?]
at java.util.stream.ReduceOps$ReduceOp.evaluateParallel(Unknown Source) ~[?:?]
at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:?]
at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:?]
at tech.pegasys.web3signer.signing.config.SignerLoader.mapToArtifactSigner(SignerLoader.java:241) ~[web3signer-signing-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.signing.config.SignerLoader.mapMetadataToArtifactSigner(SignerLoader.java:208) ~[web3signer-signing-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.signing.config.SignerLoader.load(SignerLoader.java:90) ~[web3signer-signing-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.core.Eth2Runner.loadSignersFromKeyConfigFiles(Eth2Runner.java:304) ~[web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.core.Eth2Runner.lambda$createArtifactSignerProvider$2(Eth2Runner.java:259) ~[web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.signing.config.DefaultArtifactSignerProvider.lambda$load$1(DefaultArtifactSignerProvider.java:51) ~[web3signer-signing-24.1.0.jar:24.1.0]
at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.lang.OutOfMemoryError: Java heap space
2024-01-20 01:00:54.645+00:00 | main | INFO | Runner | Web3Signer has started with TLS disabled, and ready to handle signing requests on 0.0.0.0:9000
2024-01-20 01:04:54.078+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@3e8b3b79 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.079+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@1a7cbc5f (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.080+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@1a5f30b0 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.080+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@42d6cdf9 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.081+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@15519c7c (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.081+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@6ab054cd (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.082+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@909239b (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.082+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@7dc48e46 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.082+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@646f5ffe (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:04:54.083+00:00 | ForkJoinPool.commonPool-worker-20 | WARN | PoolBase | HikariPool-1 - Failed to validate connection org.postgresql.jdbc.PgConnection@c116982 (This connection has been closed.). Possibly consider using a shorter maxLifetime value.
2024-01-20 01:08:44.428+00:00 | main | INFO | Web3SignerApp | Web3Signer has started with args --key-store-path=/data/keyfiles,–http-listen-port=9000,–http-listen-host=0.0.0.0,–http-host-allowlist=web3signer.web3signer.dappnode,brain.web3signer.dappnode,prysm.migration.dappnode,validator.prysm.dappnode,–http-cors-origins=,–metrics-enabled=true,–metrics-host,0.0.0.0,–metrics-port,9091,–metrics-host-allowlist=,–idle-connection-timeout-seconds=900,eth2,–network=mainnet,–slashing-protection-db-url=jdbc:postgresql://postgres.web3signer.dappnode:5432/web3signer-mainnet,–slashing-protection-db-username=postgres,–slashing-protection-db-password=mainnet,–slashing-protection-pruning-enabled=true,–slashing-protection-pruning-epochs-to-keep=500,–key-manager-api-enabled=true
2024-01-20 01:08:44.435+00:00 | main | INFO | Web3SignerApp | Version = web3signer/v24.1.0/linux-x86_64/-eclipseadoptium-openjdk64bitservervm-java-17
2024-01-20 01:08:45.014+00:00 | main | INFO | Eth2SubCommand | Network: mainnet
Spec Name: PHASE0, Fork Epoch: 0, First Slot: 0
Spec Name: ALTAIR, Fork Epoch: 74240, First Slot: 2375680
Spec Name: BELLATRIX, Fork Epoch: 144896, First Slot: 4636672
Spec Name: CAPELLA, Fork Epoch: 194048, First Slot: 6209536

2024-01-20 01:08:45.040+00:00 | main | INFO | HikariDataSource | HikariPool-1 - Starting…
2024-01-20 01:08:45.130+00:00 | main | INFO | HikariPool | HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@3178219a
2024-01-20 01:08:45.131+00:00 | main | INFO | HikariDataSource | HikariPool-1 - Start completed.
2024-01-20 01:08:45.275+00:00 | main | INFO | HikariDataSource | HikariPool-2 - Starting…
2024-01-20 01:08:45.283+00:00 | main | INFO | HikariPool | HikariPool-2 - Added connection org.postgresql.jdbc.PgConnection@4351ed61
2024-01-20 01:08:45.283+00:00 | main | INFO | HikariDataSource | HikariPool-2 - Start completed.
2024-01-20 01:08:45.399+00:00 | main | INFO | MetricsHttpService | Starting metrics http service on 0.0.0.0:9091
2024-01-20 01:08:45.456+00:00 | vert.x-eventloop-thread-0 | INFO | MetricsHttpService | Metrics service started and listening on 0.0.0.0:9091
2024-01-20 01:08:45.464+00:00 | pool-2-thread-1 | INFO | SignerLoader | Loading signer configuration metadata files from /data/keyfiles
2024-01-20 01:08:45.472+00:00 | pool-2-thread-1 | INFO | SignerLoader | Signer configuration metadata files read in memory 27 in 00:00:00.006
2024-01-20 01:08:45.516+00:00 | pool-2-thread-1 | INFO | SignerLoader | Converting signing metadata to Artifact Signer using parallel streams …
2024-01-20 01:08:45.517+00:00 | ForkJoinPool.commonPool-worker-15 | INFO | SignerLoader | 10 signing metadata processed
2024-01-20 01:08:45.518+00:00 | ForkJoinPool.commonPool-worker-19 | INFO | SignerLoader | 20 signing metadata processed

Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread “vertx-blocked-thread-checker”
2024-01-20 01:08:53.652+00:00 | main | ERROR | Runner | Error loading signers
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
at java.util.concurrent.FutureTask.report(Unknown Source) ~[?:?]
at java.util.concurrent.FutureTask.get(Unknown Source) ~[?:?]
at tech.pegasys.web3signer.core.Runner.run(Runner.java:121) [web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.core.Eth2Runner.run(Eth2Runner.java:401) [web3signer-core-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.commandline.subcommands.ModeSubCommand.run(ModeSubCommand.java:33) [web3signer-commandline-24.1.0.jar:24.1.0]
at picocli.CommandLine.executeUserObject(CommandLine.java:1939) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine.access$1300(CommandLine.java:145) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.handle(CommandLine.java:2352) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.handle(CommandLine.java:2314) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine$RunLast.execute(CommandLine.java:2316) [picocli-4.6.2.jar:4.6.2]
at picocli.CommandLine.execute(CommandLine.java:2078) [picocli-4.6.2.jar:4.6.2]
at tech.pegasys.web3signer.commandline.CommandlineParser.parseCommandLine(CommandlineParser.java:85) [web3signer-commandline-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.Web3SignerApp.executeWithEnvironment(Web3SignerApp.java:50) [web3signer-app-24.1.0.jar:24.1.0]
at tech.pegasys.web3signer.Web3SignerApp.main(Web3SignerApp.java:35) [web3signer-app-24.1.0.jar:24.1.0]
Caused by: java.lang.OutOfMemoryError: Java heap space
2024-01-20 01:08:53.716+00:00 | main | INFO | Runner | Web3Signer has started with TLS disabled, and ready to handle signing requests on 0.0.0.0:9000

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.