[Feature]: AutoDeploy: unit testing with node+arg counting

🚀 The feature, motivation and pitch

From time to time, we exhibit regressions related to certain transforms causing e2e failures/regressions. This can be caused by changes in other transforms or mishandling of corner cases.

A recent example of this is https://github.com/NVIDIA/TensorRT-LLM/issues/8389

This can have downstream issues for both accuracy and performances.

This ticket is tracking adding a new type of unit/e2e test to track such potential regressions. The idea is to build test infrastructure where we do "node counting" on the final graph. More specifically, we can do the following:

1. Run an e2e test like test_ad_build_and_run_single.py
2. Check out the graph in the final stages of the transform pipeline
3. Do some basic graph analysis on the final graph such as
   a. How many nodes of each type?
   b. How many function calls for each op?
   c. Check if important arguments are correct
   d. Analyze resulting shapes of fake tensors from shape prop
   e. ....

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.