• Early use to find printers and cameras in lans
  • Mobile devices such as monitors, TVS, and cell phones are found in the LAN
  • It is used to discover iot devices in the LAN iot scenario

Here is A simple network. Discovery means that PC A and PC B know that one or both of them exist.

The difference between service discovery and device discovery

  • Discover service: Discovers a port process service program of a device. A device can have multiple services.
  • Discover the service can not use the service to confirm the device?

Note: Discovering a service does not mean using a service; these are two different functions

Basic Network Knowledge

This part covers unicast, broadcast, and multicast.

  • [Widely used service discovery protocol Zeroconf](# widely used service discovery protocol Zeroconf)

unicast

Two In A local area network, the program A in PC A wants to send data packets to the program B in PC B over the computer network (the two programs interact to determine the unique destination) in the form of unicast. Most common network protocols are unicast, such as TCP.

So before unicast, program A must know program B’s IP + PROt, how to do?

  1. Know in advance

    • If the IP of PC B is static on the LAN, it can be written in program A ahead of time

  2. ask

    • If PC B’s IP is dynamic, it can be queried on the LAN.

radio

One is broadcast, where a program sends packets to all the devices in the network. The query process of program A is shown as follows:

If PC A sends A broadcast with its own address information, when PC B receives the broadcast, it will inform PC A of its own address in unicast form. If PC B uses broadcast, I only give A unicast schematic diagram here:

Multicast (multicast)

Another form of inquiry is multicast.

As can be seen from the above, broadcast will send data packets to all devices in the LAN, but PC C does not want to receive this data packet, because it is not a provider of a certain service type, and does not need to reply to the data received. Therefore, communication between PC A and PC C wastes resources (occupying communication links and one more copy of data), and multiple devices may cause broadcast storms (broadcast packets fill the entire network and other packets cannot be forwarded).

Root cause: There is no one to classify the types of services, that is, there is no grouping. If the packets are grouped, the layer 2 or 3 gateway device can forward the packets to the members of the group. The devices that are not members of the group will not receive the packets.

Therefore, The Internet Group Management Protocol (IGMP) appears on Layer 3 devices. The Layer 2 gateway uses IGMP Snooping (Internet Group Management Protocol Snooping) to implement multicast.

Ps: Of course, there are other multicast protocols, such as layer 3 PIM, MSDP, MBGP; Layer 2 multicast VLANS. (Yes, I don’t even know 🤑)

Brief introduction to IGMP

IGMP, also known as the Internet Group Management Protocol, is a Protocol used to manage IPv4 multicast members in the TCP/IP Protocol family. IGMP is used to establish and maintain multicast group membership between the receiver host and its directly adjacent multicast router. IGMP manages group members by exchanging IGMP packets encapsulated in IP packets between the receiver host and the multicast router. Multicast routers are elected and change dynamically.

IP multicast communication is characterized by the fact that packets are sent from one source and forwarded to a specific set of receivers. However, in the multicast communication model, the sender does not pay attention to the location information of the receiver and only sends the data to the agreed destination multicast address. In order for the multicast packet to reach the receiver, some mechanism is needed to enable the multicast router connected to the receiver network segment to know which multicast receivers exist on this network segment and to ensure that the receivers can join the corresponding multicast group. IGMP is a protocol used to establish and maintain multicast group membership between the receiver host and the multicast router directly adjacent to its network segment.

Source: the wiki

Zeroconf protocol mainly serves layer 3, so here is a simple procedure for layer 3 gateway devices using IGMP V3 (in my case, V3) to manage packets:

Adding a host to a Group

The host actively informs the multicast router in the network that it wants to join a certain group. At this time, the multicast router knows that there are members of group 224.0.0.251 in the network and establishes the forwarding table of the group, as shown in the following figure:

Note: all the addresses in the figure are my examples and may not meet the standard

When another host sends a broadcast packet with destination ADDRESS 224.0.0.251, the multicast router forwards the packet to other members of the group, so the source host does not care who the destination host is.

The multicast router discovers the group itself

Multicast routers (called queriers in IGMP) periodically broadcast to the LAN to check for the presence of groups, as shown in the following figure:

It can be seen that all hosts will receive the query packet from the multicast router. Assuming that PC A, PC B and PC C are all members of group 224.0.251, they will start A random timer after receiving the packet, and the one who finishes the timer first will send the multicast packet, such as PC B in the figure above. After receiving the multicast router, it knows that there are other members of this group in the network and continues to forward the group. After receiving the multicast router, other hosts do not send repeated replies. This is to reduce the occupation of the network through suppression.

Ps: In IGMPV3, the departure of a group member will be notified to the multicast router without waiting for the query.

Sends a multicast message

PC A sends the multicast to the multicast router, which forwards the multicast to the multicast members, as shown in the following figure. The source address is its own host address, and the destination address is the multicast address, but both are received and forwarded by the multicast router to the members in the group:

Implement a device discovery yourself

When you understand unicast, broadcast, multicast, to write a LAN device discovery function, it is not difficult.

Expose their

For example, you can choose a multicast implementation:

  • The host announces to the multicast router that it wants to be a member of a certain multicast. Note: The range of available multicast IP addresses is limited as follows:
“~ 224.0.0.255 For the reservedmulticastAddress (permanent group address), address 224.0.0.0 reserved for allocation, other addresses forRouting protocolUse;
224.0.1.0 to 231.255.255.255 233.0.0.0 to 238.255.255.255 A publicmulticastAddress, which can be used on the Internet;
232.0.0.0 ~ 232.255.255.255 Available to the usermulticastAddress (temporary group address), valid throughout the network;
239.0.0.0 ~ 239.255.255.255 For local managementmulticastAddress that is valid only in a specific local scope.
  • A host sends a multicast message with its own address information
  • After receiving the multicast information, members in a group can discover each other. The communication mode depends on the service scenario

Waiting for someone to find you

  • The host announces to the multicast router that it wants to be a member of a certain multicast.
  • A host sends a multicast message with its address and asks who can provide a service
  • If one of the group members happens to provide the service, unicast or multicast replies are used.

The problem

It’s great to do it yourself, but there are some problems:

  • How to determine the packet protocol asked? In this case, you do not know the IP address of the peer party but the service type provided by the peer party. You hope to find the specific IP address of the peer party based on the service type.

    If you just want to play by yourself, you can make up your own protocol, which includes the application service type, application service name, IP address and port declaration specification, so that finding each other, communicating with each other and then using each other’s services is no problem.

    But you set the agreement in the industry is not general, even if you set the agreement is more beautiful and elegant, want to become the industry standard is also more difficult things, but can do, specific query RFC related information, accidentally become the standard, blood earn!

    Are there agreements already in place in the industry? That’s when you might think of DNS services.

  • What if there is no DNS service?

    DNS services are at the application layer. You cannot guarantee that a DNS server is deployed on the network

  • What if the operating system of the device does not support DHCP?

    DHCP is at the application layer

  • What if the technical personnel does not proactively configure the hosts host table?

  • What if some operating systems do not support hosts configuration at all?

Widely used service discovery protocol Zeroconf

The more common service discovery protocol for lans is Zeroconf

Zeroconf is Zero Configuration Networking. Zeroconf was introduced in 1999 as a networking technology for automatically generating available IP addresses, without the need for additional manual configuration and a dedicated configuration server.

Zero configuration network service, as cool as it sounds, this protocol enables:

  • Automatic IP assignment
  • Provides the service discovery function that does not depend on the DNS server

Perfect solution to the previous problem

Bonjour

I’ll start with Apple’s Bonjour protocol, because Bonjour is one of the best known implementations of Zeroconf.

Bonjour’s official website:

Apple has made Bonjour open source, and in OS X and iOS, Bonjour offers the ability to advertise and discover services using Foundation, Core Foundation and C apis. In OS X, Bonjour also provides a Java API. On other platforms, such as Windows and Linux, Bonjour provides the C API.

Source: Bonjour

This means that devices with many different operating systems can easily discover each other in a LAN by connecting to the Bonjour protocol.

Apis provided by the Bonjour protocol

Bonjour hides the concrete implementation of the bottom layer and provides a simple API to the upper layer application. The user changes from three basic communications to simple function calls:

  1. Notification service API: Used to announce what services it provides
  2. Discover service API: Used to query whether there are services it needs in the LAN
  3. Parsing service apis: Obtain service instances by discovering service apis to further parse the information provided by the service

Bonjour implementation

Bonjour is implemented using three protocols: link-local Address, Multicast Dns, and DNS-BASED Service Discovery (DNS-SD), which are described one by one in the following sections.

The Link – Local Address

RFC document address

Link-local Address protocol is used to solve IP problems

Link-local address is a special type of address in a computer network. It is used only by hosts in the network segment or broadcast domain to communicate with each other. Such hosts usually do not require external Internet services, only the need for communication between hosts. IPv4 link-local addresses are defined in the 169.254.0.0/16 address block. IPv6 is defined in the FE80 ::/10 address block.

Source: the wiki

In short, you select an address in a certain range (169.254.0.0 ~ 169.254.255.255) for detection. If it is occupied in the LAN, you select it again. If it is not used, you use it. The process is shown as follows:

  1. Query using ARP Select the MAC address corresponding to the IP address and query it three times

  1. If not, announce that you want to use the IP address and bring your MAC address with you

3. If the unicast response is obtained, repeat steps 1 and 2

summary

In this way, IP allocation is realized, which feels less efficient, but should be sufficient in the LAN.

Multicast Dns protocol

RFC document address

This brings us to Zeroconf’s second function: providing service discovery without relying on a DNS server.

The Multicast Dns (abbreviated MDNS) implements Dns in the LAN through Multicast. The packet format is basically the same as Dns, but the resource record data is added. This resource record will not be discussed here and will be discussed in detail later.

Therefore, the protocol can be used to establish the relationship between the service name and IP. The service name is defined in the form of a domain name. The MDNS top-level domain name is.local

Take a look at the specific process:

To send a Multicast, you need to know the Multicast Dns address 224.0.0.251 and port 5353.

  1. The host that provides the service sends the domain name XXX. Local and the IP address of the service to the host in a multicast packet. The packet is repeated three times. The process is as follows:

  1. If no other host replies, the host sends a multicast packet to announce that it provides services. The IP address is XXXX and port is XXX. The host sends a response packet

    One important thing to note here is that hosts using MDNS maintain a mapping table and only process data with a. Local top-level domain name, and the storage area is separate from the DNS service

3. If someone replies that the domain name has been mapped, the system changes the domain name and performs steps 1 and 2 again. The response is also multicast, not unicast.

There is an interesting point here. MDNS uses multicast reply. The host can query the local cache and only need to reply to a host whose timer has expired to realize the reply suppression function mentioned in the above multicast knowledge and reduce the occupation of the network.

summary

In this way, the mapping of service domain name and IP is completed. There is a function to implement the reply suppression of IGMP protocol by using cache. I believe that I will consider using this method to avoid repeated communication when I encounter one-to-many query communication in the future.

Directory server

The Multicast DNS protocol maps service domain names to IP addresses. However, we know that a host can provide multiple services, and services can also be classified again. For example, there are many services of the print type, and tree data structures have begun to appear.

The first concept to understand is directory servers

A Directory service (English: Directory Service) is a software system that stores, organizes, and provides information access services. In software engineering, a Directory is a mapping of names and values. It allows you to look up a value based on a given name, similar to a dictionary. As in a dictionary, each word may have multiple meanings, and in a table of contents, a name may be associated with multiple different pieces of information. Similarly, just as a word can have multiple different pronunciations and multiple different meanings, a name in a directory can have multiple different types of values.

Directories may only provide a very narrow range of node types and numeric types, or they may support an arbitrary or extensible set of types. In a phone directory, the node is the name and the numeric entry is the phone number. In DNS, nodes are domain names and numeric entries are IP addresses (as well as aliases, mail server names, and so on). In a network operating system directory, nodes are resources managed by the operating system, including users, computers, printers, and other shared resources. Many directory services have been used since the advent of the Internet, but this article focuses on those from x.500

Source: the wiki

To use a directory server, you have to configure it, which does not comply with the Zeroconf protocol.

Dns-based Service Discovery (SD) protocol

RFC document address

Dns-based Service Discovery (DNS-SD) uses the three types of storage in the resource record function of MDNS to establish the mapping relationship between service types, service instances, and service instance IP addresses + PROt to achieve rapid service discovery. In a nutshell: The protocol defines three types of stored data formats, the equivalent of creating an address book, and the data formats within the address book, to facilitate the discovery of services.

You might wonder why MDNS doesn’t just implement this functionality?

In my opinion, the function definition of MDNS is clear, which is used to achieve domain name resolution without DNS service. Service discovery does not belong to his function, which belongs to a certain business scenario, so Shavadhika combines DNS-SD with MDNS to realize service discovery.

Dns-sd uses the MDNS data segment as follows:

  • PTR recording: Mapping service names to service types
  • SRV record: Map services to IP addresses and port numbers
  • TXT records: Other data attached to the service

Here’s how DNS-SD works:

Enquiry and declaration service

1. Query service

Dns-sd puts the service domain name, IP address and PROT information in the SRV area based on MDNS:

Data captured in packets can be seen as more than 200 milliseconds will send a query, check three times:

The packet data is as follows:

2. Declaration service

If no other host replies, the host uses multicast to declare that it provides services and advertises PTR response packets:

The captured packet is a PTR response packet:

Cache flush: true indicates that the PTR is flush and does not need to be updated.

  1. Service conflict

If any conflicting responses are received, repeat steps 1 and 2 to change the service domain name.

The discovery service

  1. The host periodically uses multicast to query the service list of a certain type, and sends the same PTR query. Because the query starts with a certain service type, the host then obtains the service list of a certain service type:

The content of the message. Notice that answers might have multiple services. I only have one service. :

Here you may be wondering, why are queries and replies together? This is the same RSVP suppression function mentioned above. Multicast will be sent to the multicast router first, but the multicast router has cache records, and the multicast router will directly reply to the result.

Analytical services

Also notice the absence of IP and PROT? Remember the three storage segments above? PTR query, IP and service domain name mapping relationship in SRV data segment oh!

So further service parsing is required.

My process is as follows:

  1. The device IP address is 172.17.234.158. The destination IP address is 224.0.0.251 after multicast query and service content is parsed.
  2. When 172.17.224.57 receives a unicast reply, PTR, SRV, and TXT data are displayed in the data area. 172.17.234.158 only needs to obtain the data itself to complete the service information parsing.

conclusion

Duty is very clear, can see the agreement to the grain size is very fine, Bonjour will be the same after the combination of the agreement also focus on the three functions, not provided to the upper discovered continue communication interface, even if it has the ability to provide unicast interface, after all, it is used, but has since IP + prot, how communication is to use the party.

The biggest gain this time is actually: query suppression and reply suppression these two functions, too 🐂🍺.

Note: The content of all protocols in this paper is based on the analysis of online materials and RFC documents combined with captured packet data. If I make mistakes, please point out. If you do not understand or do not approve, you can check the RFC documents by yourself.