validator: Ensure non-empty outbox on init. (#586)

validator: Ensure non-empty outbox on init.

This addresses the issue described in #575 wherein
validators crash after new contract deployment due to
failed calls on outbox.latest_checkpoint().
The calls fail due to u256 underflow on count()-1,
since count() is zero on init,
and latest_checkpoint() wants to return an
**index**, which is count() - 1.

This PR adds a retry loop just prior to entering
the existing main validator-submit-loop. I chose
to poll outbox.count() rather than try to decode
the error code or revert reason on failed calls
to latest_checkpoint() so that we avoid tightly
coupling this behavior to particular error codes
or messages, and in fact may want to preserve
exisitng behavior when calls to outbox.latest_checkpoint()
fail for other reasons.

Tested with a modified `rust/run-locally.sh` which
asks Kathy to sleep for 5 minutes prior to sending
messages. Without this change, the validator crashes.
With this change, the validator does not crash, and
after 5 minutes, starts signing checkpoints as designed.

rust/agents/validator/src/submit.rs

@@ -37,7 +37,20 @@ impl ValidatorSubmitter {
         let span = info_span!("ValidatorSubmitter");
         let reorg_period = Some(self.reorg_period);
         tokio::spawn(async move {
-            let mut current_index = self.checkpoint_syncer.latest_index().await?.unwrap_or_default();
+            // Ensure that the outbox has > 0 messages before we enter the main
+            // validator submit loop. This is to avoid an underflow / reverted
+            // call when we invoke the `outbox.latest_checkpoint()` method,
+            // which returns the **index** of the last element in the tree
+            // rather than just the size.  See
+            // https://github.com/abacus-network/abacus-monorepo/issues/575 for
+            // more details.
+            while self.outbox.count().await? == 0 {
+                info!("waiting for non-zero outbox size");
+                sleep(Duration::from_secs(self.interval)).await;
+            }
+
+            let mut current_index =
+                self.checkpoint_syncer.latest_index().await?.unwrap_or_default();
 
             info!(current_index=current_index, "Starting Validator");
             loop {