Matter (formerly Project CHIP / Connected Home over IP) Application Layer support in ESPHome?

**Describe the problem you have/What new integration you would like**

Request native support for Matter (formerly **_Project CHIP_** / **_Connected Home over IP_**) Application Layer in the ESPHome firmware.

Please add a minimal Matter application-layer framework component to ESPHome that enables building Matter end-devices using ESP32 platform over WiFi/Ethernet (communicating via IPv6 over a standard local LAN network at home) as endpoint products, with support for the Matter standard for commisioning of Matter devices.

- https://en.wikipedia.org/wiki/Matter_(standard)

Matter is a new high-level IoT device connectivity standard owned and governed by the [CSA (Connectivity Standards Alliance, formerly the Zigbee Appliance)](https://en.wikipedia.org/wiki/Connectivity_Standards_Alliance) that aims to improve interoperability and compatibility between different manufacturer using standardized security for devices that can be controlled and updated locally via non-proprietary gateway/hubs/bridges/controllers, without internet or cloud access, nor requiring users to registering any accounts at third-parties.

- https://csa-iot.org/all-solutions/matter/

The goal want to achieve here is to be able to build our own Matter compatible devices using ESPHome firmware on ESP32 chips.

* https://github.com/project-chip/connectedhomeip/blob/master/docs/platforms/esp32/setup_idf_chip.md
* https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-minimal-app/esp32
* https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-app/esp32

Note! Understand that this is a general feature request for a Matter "Application-Layer" component for ESPHome on ESP32 chips, so it is not a specific request for any of the "Network-Layer" (i.e. not the lower layers for transport, network and physical media).

The Matter application layer sits above the other layers of the protocol stack, relying on them for reliable communication.

Note that the network layer of Matter application-layer connectivity standard relies on IPv6 (IP version 6 using both TCP and UDP), so will, in theory, be able to use different transport media, including Wi-Fi, Ethernet, and Thread (OpenThread), however, as this feature request is for high-level Matter Application Layer support, so requesting that consider first scope any initial projects to focusing on basic IoT device control over just WiFi and Ethernet transport links to achieve MVP (Minimum Viable Product) functionality on one of the ESP32 SoCs that already has mature support in ESPHome and loads of resources needed for development (e.g. ESP32-C3 and ESP32-S3), and as such not yet put focus on adding Thread (OpenThread) support for Matter nor ESP32 chips with resource constraints, or at least not initially.

- https://github.com/project-chip/connectedhomeip/blob/master/README.md
- https://github.com/project-chip/connectedhomeip/blob/master/README.md#architecture-overview

Check out these Matter device code examples:

- https://github.com/project-chip/connectedhomeip/tree/master/examples
- https://github.com/project-chip/connectedhomeip/tree/master/examples/light-switch-app
- https://github.com/project-chip/connectedhomeip/tree/master/examples/lighting-app
- https://github.com/project-chip/connectedhomeip/tree/master/examples/temperature-measurement-app
- https://github.com/project-chip/connectedhomeip/tree/master/examples/pigweed-app/
- https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-minimal-app
- https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-minimal-app/all-clusters-common
- https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-app
- https://github.com/project-chip/connectedhomeip/tree/master/examples/all-clusters-app/all-clusters-common

Also check out CSA's project-chip (Matter) project's "ZAP" to generate clusters and attributes or other entities. ZAP (which stands for "ZCL Advanced Platform" ) is a generic generation engine and user interface for applications and libraries based on Zigbee Cluster Library, the specification developed by the Connectivity Standards Alliance to configure Matter (and Zigbee Pro) applications.

- https://github.com/project-chip/zap

The other incentive for wanting native Matter support in ESPHome to allow users to make DIY Matter devices is that it is likely to attract more developers and expand the userbase if implemented as once future Matter specification device type feature set have matured a bit more then upcoming versions of the Matter standard has the potential of being adopted by most other major manufacturers because it was originally founded by Amazon (developers of the Alexa/Echo ecosystem ), Apple (developers of HomeKit ecosystem), Google (developers of the Google Home/Nest/Android ecosystems), Samsung (developers of other SmartThings ecosystem), and the Zigbee Alliance, (now the Connectivity Standards Alliance), and subsequent number of members that have since joined in collaboration of this Matter standard project have been huge and include IKEA, Signify/Philips, Schneider, Siemens Eve by ABB), and perhaps more importantly for ESPHome, also MCU SoC manufacturers like Espressif, Nordic Semiconductor, and Silicon Labs.

* https://csa-iot.org/members/
* https://blog.espressif.com/matter-38ccf1d60bcd
* https://www.nordicsemi.com/Products/Technologies/Matter
* https://www.silabs.com/wireless/matter

There articles are good summaries why this "Matter" smart home standard matters or more importantly will matter in the future:

* https://www.androidpolice.com/matter-smart-home-standard-explained/
* https://www.theverge.com/23390726/matter-smart-home-faq-questions-answers
* https://www.theverge.com/22832127/matter-smart-home-products-thread-wifi-explainer
* https://arstechnica.com/gadgets/2022/10/matter-and-thread-could-fix-smart-home-compatibility-but-dont-get-excited-yet/
* https://www.pcmag.com/explainers/matter-explained

While the initial Matter standard (e.i. version 1.0) specification will take a lot of inspiration from the upcoming ZCL (Zigbee Cluster Library) specifications I understand that the very first version will still be very limited in amount of device types and attributes/features/functions it can support in the beginning, however, more device types and attributes/features/functions will be added later in each future revision/version.

As such it will in the beginnig not be close to being as flexible as just using ESPHome without the Matter protocol (or even using a other existing standard IoT protocol like Zigbee). This will limit the initial Matter 1.0 standard to at first only supporting simpler device types with control features, such as Switch or Outlet (On/Off), Lights (including dimming + color control), Fans, Window Covering, Thermostats, Locks, and basic sensors like Temperature, Humidity, Pressure, Flow, Occupancy, and Contact sensor, as well as speaker volume.

https://csa-iot.org/all-solutions/matter/matter-faq/#:~:text=Which%20device%20categories%20does%20Matter%20support%3F%C2%A0

(CSA does already have other additional device types on its roadmap and plans on introducing more in each future version, and some of those future device types mentioned are for example; Energy management, Garage door and gate controller sensors, Home security cameras, Smoke and CO detectors, other Environmental quality sensors and controls such as indoor air quality monitors and air purifiers).

Anyway, the core concept/idea behind this new Matter is a long-term future vision to end proprietary communication protocols for basic home automation devices so that users should be able to buy any Matter device from any manufacturer and use it in any third-party ecosystem from other manufacturers as long as it supports Matter. As such the main benefit of Matter being device interoperability is cross-manufacturer compatibility, as in devices that fully comply with the Matter standard will be directly supported natively by third-party ecosystems, including the most popular ones from large companies for example; Google Home / Googe Nest (Google Assistant), Amazon Echo (Amazon Alexa), and Samsung SmartThings. This would mean that if a user could configure an ESPHome-based firmware as a standard Matter device then it will be possible to commission/join/pair that device directly to any ecosystem without going through a bridge or proxy application/integration that converts it is messages to a proprietary protocols (like for example Apple's HomeKit).

- https://csa-iot.org/all-solutions/matter/ (redirected from https://buildwithmatter.com )
- https://github.com/project-chip
- https://github.com/project-chip/connectedhomeip
- https://github.com/project-chip/connectedhomeip-doc

Espressif Systems which makes ESP32 (and ESP8266) is already part of the CSA / Connectivity Standards Alliance (formerly known as the Zigbee Alliance governs the Matter/CHIP standard), with Espressif ESP32 Wi-Fi series hardware platforms will be one of the first reference platforms for Matter/CHIP, and Espressif also have announced that they will have upcoming Thread/OpenThread based ESP32 series on their roadmap soon and those will support Matter over Thread at their launch.

- https://www.espressif.com/en/news/Matter_on_ESP32
- https://www.espressif.com/en/news/Matter_Series_Blogposts
- https://zigbeealliance.org/member/espressif-systems/

For more details on exactly how developers can use Matter on ESP32 read Hrishikesh Dhayagude's blogpost on the ESP Journal:

- https://blog.espressif.com/announcing-matter-previously-chip-on-esp32-84164316c0e3
- https://medium.com/the-esp-journal/announcing-matter-previously-chip-on-esp32-84164316c0e3

FYI, Home Assistant developers have been experimenting with the Matter standard on ESP32 (though not based on ESPHome):

https://www.home-assistant.io/integrations/matter#experiment-with-matter-using-a-esp32-dev-board

Also see:

- https://blog.espressif.com/what-does-matter-mean-to-you-fa3bb53a7793
- https://blog.espressif.com/matter-clusters-attributes-commands-82b8ec1640a0
- https://blog.espressif.com/matter-device-to-device-automations-bdbb32365350
- https://blog.espressif.com/matter-bridge-for-non-matter-devices-d3b7f003a004

While still a work-in-progress you can find the ESP32 code in the Matter/CHIP SDK here:

- https://github.com/project-chip/connectedhomeip/search?q=esp32

**Please describe your use case for this integration and alternatives you've tried:**

**Additional context**

Not sure if it is still the case but Project CHIP initially mandated the use of IPv6 addresses, so not sure if IPv6 support will still be a hard requirement or if when the 1.0 version of the Matter Application Layer specification is released will also support Matter over IPv4. Also, see IPv6 support request -> https://github.com/esphome/feature-requests/issues/718 (EDIT: Apparently this might no longer be applicable since IPv6 is supported on ESP32 and Matter/CHIP will officially only be supported on ESP32 anyway and no upstream support for ESP8266/ESP82xx?).

# Architecture Overview

_Matter aims to build a universal IPv6-based communication protocol for smart home devices. The protocol defines the application layer that will be deployed on devices and the different link layers to help maintain interoperability. The following diagram illustrates the normal operational mode of the stack:_


_The architecture is divided into layers to help separate the different responsibilities and introduce a good level of encapsulation among the various pieces of the protocol stack. The vast majority of interactions flow through the stack captured in the following Figure:_


1. _**Application:** High-order business logic of a device. For example, an
application that is focused on lighting might contain logic to handle turning
on/off the bulb as well as its color characteristics._

2) _**Data Model:** The data layer corresponds to the data and verb elements that
help support the functionality of the application. The Application operates
on these data structures when there is an intent to interact with the device._

3. _**Interaction Model:** The Interaction Model layer defines a set of
interactions that can be performed between a client and server device. For
example, reading or writing attributes on a server device would correspond to
application behavior on the device. These interactions operate on the
elements defined at the data model layer._

4) _**Action Framing:** Once an action is constructed using the Interaction
Model, it is serialized into a prescribed packed binary format to encode for
network transmission._

5. _**Security:** An encoded action frame is then sent down to the Security Layer
to encrypt and sign the payload to ensure that data is secured and
authenticated by both sender and receiver of a packet._

6. _**Message Framing & Routing:** With an interaction encrypted and signed, the
Message Layer constructs the payload format with required and optional header
fields; which specify the message's properties and some routing information._

7) _**IP Framing & Transport Management:** After the final payload has been
constructed, it is sent to the underlying transport protocol for IP
management of the data._

PS: Off-topic but FYI, Thread (OpenThread) devices been proven to be faster than Zigbee and Bluetooth Multicast (Bluetooth Mesh):

https://www.reddit.com/r/homeautomation/comments/nxmehn/clearing_up_confusion_thread_is_much_faster_than/

Start with connectedhomeip/docs/platforms/esp32/setup_idf_chip.md and the examples/all-clusters-minimal-app/esp32 example, then compare the requested scope with ESPHome's existing ESP32 support. Done would require an agreed minimal Matter application-layer scope, implementation boundaries for WiFi and Ethernet, and validation against a supported Matter device example.

Written by the indexing model from the issue text.