HF simulation resumed from a restart file has weird matrix sizes

Describe the bug
While testing a large-scale hydraulic fracture model, I encountered an error after resuming from a restart file. The code appeared to resume without any apparent errors. During the next application of the HF solver, the numerical solution appeared to be going fine and identified faces that needed to be opened. After this point however, the solver reported the following error:

ERROR: Amesos NumericFactorization failed... dumping relevant matrix for post-mortem

To Reproduce
I found this error while working with one of the new HF examples in #990 . Some key pieces of information:

• The model ran on the quartz debug partition for 30 minutes with x=4, y=2, z=4 partitions
• The fracture is propagating along a partition boundary
• The restart file was written by triggering a HaltEvent at the 28 minute mark.

I'll try to find a smaller version of this problem that shows this error, and play around with different partitioning. The following are segments of the log file and matrix dump file:

Log file after restart:
Max threads: 2
MKL max threads: 2
real64 is alias of double
localIndex is alias of long
globalIndex is alias of long long
Loading restart file /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055
Rank 0: rankFilePattern = /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_%07d.hdf5
Rank 0: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000000.hdf5
Rank 1: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000001.hdf5
Rank 2: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000002.hdf5
Rank 3: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000003.hdf5
Rank 4: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000004.hdf5
Rank 5: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000005.hdf5
Rank 7: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000007.hdf5
Rank 9: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000009.hdf5
Rank 10: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000010.hdf5
Rank 12: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000012.hdf5
Rank 13: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000013.hdf5
Rank 14: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000014.hdf5
Rank 15: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000015.hdf5
Rank 17: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000017.hdf5
Rank 19: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000019.hdf5
Rank 20: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000020.hdf5
Rank 6: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000006.hdf5
Rank 8: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000008.hdf5
Rank 11: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000011.hdf5
Rank 16: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000016.hdf5
Rank 18: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000018.hdf5
Rank 21: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000021.hdf5
Rank 22: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000022.hdf5
Rank 23: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000023.hdf5
Rank 24: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000024.hdf5
Rank 25: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000025.hdf5
Rank 26: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000026.hdf5
Rank 27: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000027.hdf5
Rank 28: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000028.hdf5
Rank 29: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000029.hdf5
Rank 30: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000030.hdf5
Rank 31: Reading in restart file at /p/lustre2/sherman/HFTS/GEOS_vs_GEOSX_benchmarks/single_fracture_clean_fluid/GEOSX_new/tmp_restart_000000055/rank_0000031.hdf5
GEOS must be configured to use Python to use parameters, symbolic math, etc. in input files
Adding Mesh: InternalMesh, mesh1
Adding Geometric Object: Box, perf_a
Adding Geometric Object: Box, source_a
Adding Geometric Object: ThickPlane, fracturable_a
Adding Solver of type Hydrofracture, named hydrofracture
Adding Solver of type SolidMechanicsLagrangianSSLE, named lagsolve
Adding Solver of type SinglePhaseFVM, named SinglePhaseFlow
Adding Solver of type SurfaceGenerator, named SurfaceGen
Adding Output: Silo, siloOutput
Adding Output: Restart, restartOutput
Adding Event: SoloEvent, preFracture
Adding Event: PeriodicEvent, outputs
Adding Event: PeriodicEvent, solverApplications_a
Adding Event: PeriodicEvent, solverApplications_b
Adding Event: PeriodicEvent, solverApplications_c
Adding Event: HaltEvent, restarts
TableFunction: flow_rate
TableFunction: sigma_xx
TableFunction: sigma_yy
TableFunction: sigma_zz
TableFunction: init_pressure
TableFunction: bulk_modulus
TableFunction: shear_modulus
Adding Object CellElementRegion named Region2 from ObjectManager::Catalog.
Adding Object FaceElementRegion named Fracture from ObjectManager::Catalog.
Running simulation
The restart-file was written during step 5 of the event loop. Resuming from that point.
Time: 184s, dt:4s, Cycle: 55
Time: 188s, dt:2s, Cycle: 56
Attempt: 0, NewtonIter: 0 ;
( Rfluid, Rsolid ) = (4.84e-01, 1.99e-13) ;

Attempt:  0, NewtonIter:  1 ; 

( Rfluid, Rsolid ) = (5.40e-02, 4.00e-09) ; Last LinSolve(iter,res) = ( 0, 0.00e+00) ;

Attempt:  0, NewtonIter:  2 ; 

( Rfluid, Rsolid ) = (2.86e-04, 4.84e-10) ; Last LinSolve(iter,res) = ( 0, 0.00e+00) ;

Attempt:  0, NewtonIter:  3 ; 

( Rfluid, Rsolid ) = (5.47e-07, 2.09e-12) ; Last LinSolve(iter,res) = ( 0, 0.00e+00) ;
++ Fracture propagation. Re-entering Newton Solve.
Attempt: 0, NewtonIter: 0 ;
( Rfluid, Rsolid ) = (1.30e-02, 1.37e-01) ;
ERROR: Amesos NumericFactorization failed... dumping relevant matrix for post-mortem

Attempt:  0, NewtonIter:  1 ; 

( Rfluid, Rsolid ) = (1.10e-02, 1.37e-01) ; Last LinSolve(iter,res) = ( 0, 0.00e+00) ;

ERROR: Amesos NumericFactorization failed... dumping relevant matrix for post-mortem
Attempt: 0, NewtonIter: 2 ;
( Rfluid, Rsolid ) = (1.10e-02, 1.37e-01) ; Last LinSolve(iter,res) = ( 0, 0.00e+00) ;

Last few lines from the file amesos-failure.dat:
79 78 -2.3811047519983836e+07
80 68 -3.3696905051534642e+06
80 63 -6.6849689075721288e+06
80 64 -3.3152784024186642e+06
80 71 -1.4833076419058008e+118
80 77 -3.3696905051534586e+06
80 72 -1.4833076419058008e+118
80 80 -1.4833076419058020e+118
80 81 -1.4833076419058020e+118
81 63 -1.0152750673919858e+07
81 64 -2.0355044939432088e+07
81 69 -1.0202294265512232e+07
81 71 -1.1691518144325470e+118
81 72 -1.1691518144325470e+118
81 78 -1.0202294265512213e+07
81 80 -1.1691518144325479e+118
81 81 -1.1691518144325479e+118