developmentseed / developmentseed/megaseg

Issue #1: Project Description and Resources

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
5
Forks
0
PR merge metrics
No merged PRs in 30d

Description

# Issue 1

## Background

Problem: Most existing deep learning architectures cannot operate on megapixel images because of computational and memory constraints of CNNs.

Solution/Research Goal: We'd like to take advantage of recent research that uses Transformers for image classification and object detection to reduce the memory footprint of features learned for segmentation. Ultimately we'd like to be able to train a model end-to-end without chipping imagery into arbitrary 512x512 squares that discard crucial context or force the network to train full passes on uninformative chips, enabling (Mega)pixel (Seg)mentation!

[DETR](https://github.com/facebookresearch/detr/tree/main/d2) is potentially a good starting point, though there may be others that are better.

### Communication

**Internal communication**:

- Issues in this github repo https://github.com/developmentseed/megaseg/
- Weekly Thursday video meetings
- potentially a slack channel if needed later

## Notes Documents

- [running research meeting notes](https://paper.dropbox.com/doc/Research-with-Freddie-Notes--BbseYOp7hpoOOMefa~kWu1b5Ag-6OcXqfNxpyTIc0b7x25qw)
- https://arxiv.org/abs/1905.03711 - Processing Megapixel Images with Deep Attention-Sampling Models
- https://arxiv.org/pdf/2010.11929.pdf - An Image is worth 16x16 Words: Transformers for Image Recognition at Scale
- https://www.youtube.com/watch?v=T35ba_VXkMY - End to End Object Detection with Transformers

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.