Backfill jobs which are lost while Dr.Elephant was down

Nobody has claimed this yet.

Dr.Elephant periodically analyses historical Mapreduce,Spark and other jobs running on a YARN cluster. To fetch a list of YARN applications, Dr.Elephant contacts YARN Resource Manager(RM) periodically to get a list of completed applications.
However, the number of completed applications retained in YARN RM is limited by below configurations:
1. “yarn.resourcemanager.max-completed-applications”
2. “yarn.resourcemanager.state-store.max-completed-applications”

As Dr.Elephant server does not run in high availability mode, it is possible that some of the applications which were not analyzed by Dr.Elephant while it was down, are not analyzed even upon restart as the applications are no longer present in YARN RM. Possible if the process is down for long.
This requirement is to overcome this shortcoming and allow jobs to be analyzed even if they are not present in RM, contingent upon whether they can be fetched by respective fetchers, after restart.

Start by examining how Dr. Elephant contacts the YARN Resource Manager and how the respective MapReduce and Spark fetchers discover applications after restart. Define a recovery path for applications no longer retained by YARN, and verify that previously missed jobs can be fetched and analyzed after downtime.

Written by the indexing model from the issue text.