Roadmap – feedback welcome!

TensorFlow Graphics was first released at Google I/O in 2019, and since, the library grew a lot, both in terms of core components, but also in terms of infrastructure which was put in place to facilitate collaboration with the community. We are excited to share with you what we believe would be great next steps for TensorFlow Graphics, and would love to hear your thoughts & suggestions!

Top level goals

• Build standard structures and implementations for differentiable 3D and graphics development.
• Provide a central place to access tooling and implementations.
• Make it easier for research results to be implemented and replicated.

Our current efforts focus in the following areas:

• A Distill.pub article on the importance of differentiable graphics and dynamics.
• Packaging mainstream 3D datasets to support research.
  • Examples include ModelNet, ShapeNet, Pix3D, etc.
• Providing complete implementations of state of the art papers.
  • CvxNet, PointNet, Neural Voxel Renderer, Point-based convolutions, and more!
• More Colab demos!
• Begin Google Summer of Code projects!
  • Implicit representations using Occupancy Networks, DepSDF and IM-NET.
  • Point-based convolutions.
  • Mesh R-CNN implementation.
• Infrastructure work.
  • Google <-> Github pipeline to facilitate open-source development.
  • Pip packaging of our differentiable rasterizer.

Completed

• Launch of a Special Interest Group! Read our approved charter to read more about this effort. If interested, you can browse the slides and watch the recording of the meeting.
• Launched a submodules folder for the community to raise awareness about their work on the themes of differentiable 3D and graphics. The first submodule is redner!
• A complete deployment pipeline between the Github and the Google infrastructure!
• Launched a projects folder to host implementations of your work! The first project is PointNet 1.0!
• Started implementing loaders for mainstream datasets. The first datasets available as TF datasets are modelnet40 and shapenet!
• Added several losses and metrics.
• Implemented differentiable triangle mesh sampler.
• Added matting Colab demo.
• Implemented trilinear interpolation.
• Implemented linear blend skinning.

This is a broad roadmap discussion rather than a scoped code task, and it mentions datasets, projects, notebooks, and infrastructure without naming a specific file or test. Start by reviewing the proposed goals and linked areas such as tensorflow_graphics/projects, datasets, and notebooks. Done would require turning one agreed priority into a separate, actionable issue.