1

I’m running a private Proof-of-Authority (PoA) Ethereum fork using Geth with the Clique consensus algorithm and a custom chainId (1698369). My setup consists of four servers:

  1. Public RPC node behind NGINX.
  2. Private RPC + 3 validators (on different ports).
  3. Private RPC + 3 validators (on different ports).
  4. Private RPC + 3 validators (on different ports).

All 13 services (1 public RPC, 3 private RPCs, 9 validators) start successfully from genesis.json and show as Active in systemctl. However, when querying eth.blockNumber on any RPC or validator node, it remains stuck at 1, even after hours of runtime.

To debug, I inspected logs for one validator with:

journalctl -u geth-mainnet-validator1.service -n 50000 | grep -E "Block|seal|mine|Commit|unlock"

Here’s a sample of the output across validators:

Mar 21 01:10:53 node2 test-geth-mainnet-validator1[139532]: INFO [03-21|01:10:53.493] Commit new sealing work number=1 sealhash=d264b8..b8970e txs=0 gas=0 fees=0 elapsed="148.258µs" Mar 21 01:10:53 node2 test-geth-mainnet-validator1[139532]: WARN [03-21|01:10:53.493] Block sealing failed err="unauthorized signer" Mar 21 01:13:28 node2 test-geth-mainnet-validator1[139532]: INFO [03-21|01:13:28.162] Commit new sealing work number=2 sealhash=f12875..f24402 txs=0 gas=0 fees=0 elapsed="115.696µs" Mar 21 01:13:28 node2 test-geth-mainnet-validator1[139532]: WARN [03-21|01:13:28.162] Block sealing failed err="unauthorized signer" Mar 21 01:11:03 node2 test-geth-mainnet-validator3[139562]: INFO [03-21|01:11:03.659] Successfully sealed new block number=1 sealhash=bfc8a4..647af4 hash=77a147..1771e5 elapsed=7.713ms Mar 21 01:11:03 node2 test-geth-mainnet-validator3[139562]: INFO [03-21|01:11:03.660] Commit new sealing work number=2 sealhash=1310ab..2ea0d8 txs=0 gas=0 fees=0 elapsed="543.651µs" Mar 21 01:11:03 node2 test-geth-mainnet-validator3[139562]: WARN [03-21|01:11:03.660] Block sealing failed err="signed recently, must wait for others"

The logs show validators attempting to seal blocks, but most fail with err="unauthorized signer", except for one instance where validator3 seals block 1. After that, no further blocks are produced, and I see err="signed recently, must wait for others".

Here’s my genesis.json:

{ "config": { "chainId": 1698369, "homesteadBlock": 0, "eip150Block": 0, "eip155Block": 0, "eip158Block": 0, "clique": { "period": 5, "epoch": 30000 } }, "nonce": "0x0", "timestamp": "0x67d1ddfd", "extraData": "0x00000000000000000000000000000000000000000000000000000000000000008582102B6e433AEb76B4baA2247c8BB67305405644C41862AFe35E7ffA5d46D106E78e56282106D24C0F541D9e0b6026dcee2532F778dB15E52AA716C17CfBaa87Ec82a26c2Ed9e0206aBF8c74d39d21eDDC1aD264D782A598DeB424d284DA751b0237eE44E3bA47fB9c036e3f9441F8c817d58f0d714c7FC79DE6A1eefAA4325B71590585B4b056B0750e97CEb07760b2b9797b7e31cfd648F7302925c28d582C1EB859B739829ea7D3B99f4445710EfBED2017F7bCeae4a6B59e451f97C6D91BA7115C8ed0c00d68a87bA8cb51aC05422138f639171d36f831F27c0000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x989680", "difficulty": "0x1", "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "coinbase": "0x0000000000000000000000000000000000000000", "alloc": { "8582102B6e433AEb76B4baA2247c8BB673054056": {"balance": "180000000000000000000000"}, "44C41862AFe35E7ffA5d46D106E78e56282106D2": {"balance": "180000000000000000000000"}, "4C0F541D9e0b6026dcee2532F778dB15E52AA716": {"balance": "180000000000000000000000"}, "C17CfBaa87Ec82a26c2Ed9e0206aBF8c74d39d21": {"balance": "180000000000000000000000"}, "eDDC1aD264D782A598DeB424d284DA751b0237eE": {"balance": "180000000000000000000000"}, "44E3bA47fB9c036e3f9441F8c817d58f0d714c7F": {"balance": "180000000000000000000000"}, "C79DE6A1eefAA4325B71590585B4b056B0750e97": {"balance": "180000000000000000000000"}, "CEb07760b2b9797b7e31cfd648F7302925c28d58": {"balance": "180000000000000000000000"}, "2C1EB859B739829ea7D3B99f4445710EfBED2017": {"balance": "180000000000000000000000"}, "F7bCeae4a6B59e451f97C6D91BA7115C8ed0c00d": {"balance": "1361000000000000000000000"}, "d2DEBaecF0591Ab97a4e1e8214fd357390f1879C": {"balance": "0"}, "68a87bA8cb51aC05422138f639171d36f831F27c": {"balance": "99997019000000000000000000000"} } }
Improve this question

1 Answer 1

Reset to default
1

I believe I see a problem.. All your validators are signing genesis.. only one should.. Try this might fix it..

Static Enodes: Copy existing nodekeys from data-mainnet-*/ to a backup dir (e.g., ~/go-ethereum/nodekey-backup/)—to preserve original enodes:

mkdir -p ~/go-ethereum/nodekey-backup cp ~/go-ethereum/data-mainnet-*/nodekey ~/go-ethereum/nodekey-backup/

Service Update: Add --nodekey to all 13 systemd services using the backup keys, used full enode list in --bootnodes:

Example for validator1: ExecStart=/home/superuser/go-ethereum/build/bin/geth \ --nodekey /home/superuser/go-ethereum/data-mainnet-validator1/nodekey \ --bootnodes "enode://dcdc208...@IP:30303,..."

New genesis: Change you genesis to this (you have an error in the extraData):

 "config": { "chainId": 698369, "homesteadBlock": 0, "eip150Block": 0, "eip155Block": 0, "eip158Block": 0, "clique": { "period": 5, "epoch": 30000 } }, "nonce": "0x0", "timestamp": "0x67dda630", "extraData": "0x0000000000000000000000000000000000000000000000000000000000000000c79de6a1eefaa4325b71590585b4b056b0750e97ceb07760b2b9797b7e31cfd648f7302925c28d582c1eb859b739829ea7d3b99f4445710efbed2017f7bceae4a6b59e451f97c6d91ba7115c8ed0c00d68a87ba8cb51ac05422138f639171d36f831f27c4c0f541d9e0b6026dcee2532f778db15e52aa7168582102b6e433aeb76b4baa2247c8bb673054056c17cfbaa87ec82a26c2ed9e0206abf8c74d39d21eddc1ad264d782a598deb424d284da751b0237ee0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", "gasLimit": "0x989680", "difficulty": "0x1", "mixHash": "0x0000000000000000000000000000000000000000000000000000000000000000", "coinbase": "0x0000000000000000000000000000000000000000", "alloc": { "c79de6a1eefaa4325b71590585b4b056b0750e97": {"balance": "180000000000000000000000"}, "ceb07760b2b9797b7e31cfd648f7302925c28d58": {"balance": "180000000000000000000000"}, "2c1eb859b739829ea7d3b99f4445710efbed2017": {"balance": "180000000000000000000000"}, "f7bceae4a6b59e451f97c6d91ba7115c8ed0c00d": {"balance": "180000000000000000000000"}, "68a87ba8cb51ac05422138f639171d36f831f27c": {"balance": "180000000000000000000000"}, "4c0f541d9e0b6026dcee2532f778db15e52aa716": {"balance": "180000000000000000000000"}, "8582102b6e433aeb76b4baa2247c8bb673054056": {"balance": "180000000000000000000000"}, "c17cfbaa87ec82a26c2ed9e0206abf8c74d39d21": {"balance": "180000000000000000000000"}, "eddc1ad264d782a598deb424d284da751b0237ee": {"balance": "180000000000000000000000"}, "44e3ba47fb9c036e3f9441f8c817d58f0d714c7f": {"balance": "1361000000000000000000000"}, "d2debaecf0591ab97a4e1e8214fd357390f1879c": {"balance": "0"}, "44c41862afe35e7ffa5d46d106e78e56282106d2": {"balance": "99997019000000000000000000000"} } }

Wipe Chain: Cleared geth/ dirs, reinited:

rm -rf ~/go-ethereum/data-mainnet-*/geth for dir in data-mainnet-*; do geth --datadir $dir init genesis.json; done

Staggered Start: Launched validator 1 (60s), then validator 2 (60s), then all 7 others—synced one block 1:

systemctl start test-geth-mainnet-validator1.service; sleep 60 systemctl start test-geth-mainnet-validator2.service; sleep 60 systemctl start test-geth-mainnet-*.service

Sent Tx: Triggered rotation:

geth attach data-mainnet-validator1/geth.ipc > eth.sendTransaction({from: "0xc79d...", to: "0xc79d...", value: web3.toWei(1, "ether"), nonce: 0})

That should help you fix the problem..

Improve this answer

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.