Optimistic block production due to RPC error

Hi everyone!

I need help to get my mainnet validators running again. I do not know for sure the root cause, but I can start with what happened:
After a normal restart of my DAppNode Extreme, my UI told me I needed to re-open the ports / enable UPnP again. I do not know why they closed in the first place, but I got UPnP enabled again and a consecutive “no NAT loopback” message appeared which persists until today. Since I get access via my own WIFI, this is not a real issue for me.

However, the following is troubling me:
Since then, my Geth client does not sync, my prysm client does.
My web3signer logs report this:

But looking at my prysm logs, the following messages appear:
My validators say my node is producing optimistic blocks due to a rpc error and therefore cannot request validation requests:

My beacon chain node shows the following:

My geth client seems to be waiting for the right prysm inputs, because now the beacon chain backfill keeps failing:

I hope I provided enough info, I am greatful for any help!

Before filing a new topic, please provide the following information.

Core DAppNode Packages versions

  • bind.dnp.dappnode.eth: 0.2.6
  • core.dnp.dappnode.eth: 0.2.64
  • dappmanager.dnp.dappnode.eth: 0.2.59, commit: 26522964
  • https.dnp.dappnode.eth: 0.1.4
  • ipfs.dnp.dappnode.eth: 0.2.18
  • vpn.dnp.dappnode.eth: 0.2.8, commit: f9a8743e
  • wifi.dnp.dappnode.eth: 0.2.8

System info

  • dockerComposeVersion: 1.25.5
  • dockerServerVersion: 19.03.8
  • dockerCliVersion: 19.03.8
  • os: debian
  • versionCodename: bullseye
  • architecture: amd64
  • kernel: 5.9.0-1-amd64
  • Disk usage: 50%

UPDATE:
For those interested, I solved the mystery myself. As I already assumed, I am a terrible analyser and thought the mistake to lay in the Prysm client, but actually it turned out to be Geth. Web3signer and internet is working fine as well. I completely discarded Geth and its files, then reinstalled from scratch. 1 Day of waiting and now I am validating again.

Once again, thank you DAppNode for making this a reality!

1 Like

Thank you for posting your follow up with the solution. I’ve been experiencing some similar “weird” (for lack of a better, or more technical term) network or connectivity issues / errors that I’ve struggled to troubleshoot on my own.

I was afraid of just starting to blow things away, but might attempt your path if i’m still unable to make any progress. Glad you’re back in business. Stay safe.