Photo by Oscar Ivan Esquivel Arteaga on Unsplash


This article from the live streaming cluster SRS’s official wiki (https://github.com/ossrs/srs/wiki/v4_CN_K8s), by founding the author Yang Chengli authorization of SRS.


By Yang Jiancheng


When SRS meets K8s: Quickly build a high concurrency live cluster


This chapter describes how to build Origin Cluster based on K8s to support the scenario of super multi-push flow.

The Origin Cluster can query the location of a stream when there is no stream on the local site by configuring the information of other source sites, and then redirect the stream to the specified source site through RTMP302. For details, see #464. The main application scenarios are as follows:

  • Source site Dr: Even if the number of streams is small, you can use two source sites to distribute the streams to different source sites, preventing all streams from being affected when the source site is faulty.
  • Massive stream pushing: A single source station can support 1000 to 3000 streams. Streams with high bit rate support fewer channels, and streams with DVR and HLS support even fewer channels. Multiple source stations in a source station cluster can receive streams at the same time and support streams of 10K to 100K.
  • Complex source station service: In addition to supporting stream push and pull, the source station also has important functions such as DVR, transcoding, and HLS. DVR and HLS involve disks, while transcoding involves CPU, which are resource dependence prone to bottlenecks, and the source station cluster expansion capability is stronger.


In this scenario, compare K8s with traditional usage:


SRS and Nginx are not in the same Pod or Node, so you need to create a Persistent Volume (PV). You can purchase NAS for example:

  • Driver type: Alicloud/NAS
  • Mount point (PV Server), which can be created, viewed, and copied on the console: 1abb5492f7-ubq80.cn-beijing.nas.aliyuncs.com
  • NFS version (PV VERS) : 3


PV and PVC can be created on the basis of NAS:

  • Pv-nas, PVS created from NAS storage, support multi-write and multi-read, Pod will recycle the storage after no use, that is, delete the data.
  • Pvc-nas, SRS and Nginx source stations for use with read and write permissions. Read SRS static files and HLS and distribute them.
cat <<EOF | kubectl apply -f -apiVersion: v1kind: PersistentVolumemetadata: name: pv-nas labels:   pv: nfs-pvspec: capacity:   storage: 100Gi storageClassName: nas accessModes:    -ReadWriteMany    -ReadOnlyMany persistentVolumeReclaimPolicy: Retain flexVolume:   driver: "alicloud/nas"   options:     server: "1abb5492f7-ubq80.cn-beijing.nas.aliyuncs.com"      path: "/k8s"     vers: "3"     options: "nolock,tcp,noresvport" --- apiVersion: v1kind: PersistentVolumeClaimmetadata: name: pvc-nasspec: accessModes:    -ReadWriteMany storageClassName: nas resources:   requests:     storage: 100Gi selector:   matchLabels:     pv: nfs-pvEOFCopy the code

Note: Please replace the mount point (PV Server) above with yours.


Step 2: Create SRS source site cluster and Nginx source site applications and services.

  • Srs-origin-config: Creates a configuration k8S ConfigMap that stores the configuration file used by the SRS Origin Server.
  • Socs: Creates a Headless service, k8S Service, and provides the Origin service based on the Headless service. Each Origin has its own service address, for example, srs-origin-0.socs, which can be invoked by the internal Edge Server.
  • Srs-origin: Create a stateful application k8S StatefulSet, run the SRS Origin Cluster, and write HLS to the shared storage PV.
  • Nginx-origin-deploy: Create a stateless application K8S deployment, run nginx, write SRS static files to PV, and read HLS and static files from shared storage PV.
  • Srs-http-service: creates a k8S service to provide HTTP services based on SLB, and Nginx provides HLS services externally.
  • The SRS – API – service: create a service k8s service, provide the HTTP service based on SLB, SRS first source provide API service station, labeled statefulset. Kubernetes. IO/pod – name: the SRS – origin – 0.
cat <<EOF | kubectl apply -f -apiVersion: v1kind: ConfigMapmetadata: name: srs-origin-configdata: srs.conf: |-   listen              1935;    max_connections     1000;   daemon              off;   http_api {       enabled         on;       listen          1985;    }   http_server {       enabled         on;       listen          8080;    }   vhost __defaultVhost__ {       cluster {           origin_cluster  on;           coworkers       srs-origin-0.socssrs-origin-1.socs srs-origin-2.socs;       }       http_remux {           enabled     on;       }       hls {           enabled         on;       }    } --- apiVersion: v1kind: Servicemetadata: name: socsspec: clusterIP: None selector:   app: srs-origin ports:  -name: socs-1935-1935   port: 1935   protocol: TCP   targetPort: 1935 --- apiVersion: apps/v1kind: StatefulSetmetadata: name: srs-origin labels:   app: srs-originspec: serviceName: "socs" replicas: 3 selector:   matchLabels:     app: srs-origin template:   metadata:     labels:       app: srs-origin   spec:     volumes:     - name: cache-volume       persistentVolumeClaim:         claimName: pvc-nas     - name: config-volume       configMap:         name: srs-origin-config     containers:     - name: srs       image: ossrs/srs:3       imagePullPolicy: IfNotPresent       ports:       - containerPort: 1935       - containerPort: 1985       - containerPort: 8080       volumeMounts:       - name: cache-volume         mountPath: /usr/local/srs/objs/nginx/html         readOnly: false       - name: config-volume         mountPath: /usr/local/srs/conf --- apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-origin-deploy labels:   app: nginx-originspec: replicas: 1 selector:   matchLabels:     app: nginx-origin template:   metadata:     labels:       app: nginx-origin   spec:     volumes:     - name: cache-volume       persistentVolumeClaim:         claimName: pvc-nas     containers:     - name: nginx       image: nginx       imagePullPolicy: IfNotPresent       ports:       - containerPort: 80       volumeMounts:       - name: cache-volume         mountPath: /usr/share/nginx/html         readOnly: true     - name: srs-cp-files       image: ossrs/srs:3       imagePullPolicy: IfNotPresent       volumeMounts:       - name: cache-volume         mountPath: /tmp/html         readOnly: false       command: ["/bin/sh"]       args: ["-c"."cp -R ./objs/nginx/html/* /tmp/html/; sleepinfinity"] --- apiVersion: v1kind: Servicemetadata: name: srs-http-servicespec: type: LoadBalancer selector:   app: nginx-origin ports:  -name: nginx-origin-service-80-80   port: 80   protocol: TCP   targetPort: 80 --- apiVersion: v1kind: Servicemetadata: name: srs-api-servicespec: type: LoadBalancer selector:   statefulset.kubernetes.io/pod-name: srs-origin-0 ports:  -name: srs-api-service-1985-1985   port: 1985   protocol: TCP    targetPort:1985EOFCopy the code

Remark: If the Replicas of srS-Origin is 2, two source stations srs-origin-0.socs and srs-origin-1.socs will be generated. If the new source station is added, e.g., Replicas is 3, You need to add srS-origin-2.socs to the configuration.


Remark: We start a container for SRS-CP-files and copy the default SRS files, refer to #1603.


Note: The OriginServer provides the streaming media source service in a cluster. The internal domain names are srS-origin-0. socs and SRS-Origin-1. socs. EdgeServer connects to the OriginServer through this domain name.


Note:Nginx reads slices generated by SRSOrigin through SharedVolume(PV) to provide HLS service.


Note: Here we choose ACK to automatically create SLB and EIP, or you can manually specify SLB, refer to specify purchase SLB and EIP.


Step 3: Create SRS edge configurations, applications, and services.

  • Srs-edge-config: Creates a k8S ConfigMap, which stores the configuration files used by the SRS Edge Server.
  • Srs-edge-deploy: Create a stateless K8S Deployment application that runs multiple SRS Edge Servers.
  • Srs-edge-service: creates a service. K8s service Provides external streaming media services based on SLB.
cat <<EOF | kubectl apply -f -apiVersion: v1kind: ConfigMapmetadata: name: srs-edge-configdata: srs.conf: |-   listen              1935;   max_connections     1000;   daemon              off;   http_api {       enabled         on;       listen          1985;    }   http_server {       enabled         on;       listen          8080;    }   vhost __defaultVhost__ {       cluster {           mode            remote;           origin          srs-origin-0.socssrs-origin-1.socs srs-origin2.socs;       }       http_remux {           enabled     on;       }    } --- apiVersion: apps/v1kind: Deploymentmetadata: name: srs-edge-deploy labels:   app: srs-edgespec: replicas: 4 selector:   matchLabels:     app: srs-edge template:   metadata:     labels:       app: srs-edge   spec:     volumes:     - name: config-volume       configMap:         name: srs-edge-config     containers:     - name: srs       image: ossrs/srs:3       imagePullPolicy: IfNotPresent       ports:       - containerPort: 1935       - containerPort: 1985       - containerPort: 8080       volumeMounts:       - name: config-volume         mountPath: /usr/local/srs/conf --- apiVersion: v1kind: Servicemetadata: name: srs-edge-servicespec: type: LoadBalancer selector:   app: srs-edge ports:  -name: srs-edge-service-1935-1935   port: 1935   protocol: TCP   targetPort: 1935  -name: srs-edge-service-8080-8080   port: 8080   protocol: TCP   targetPort: 8080EOFCopy the code

Remark: If the Replicas of srS-Origin is 2, two source stations srs-origin-0.socs and srs-origin-1.socs will be generated. If the new source station is added, e.g., Replicas is 3, You need to add srS-origin-2.socs to the configuration.


Note:Edge Server is configured to connect to OriginServer via srs-origin-0.socs, etc.


Note: Here we choose ACK to automatically create SLB and EIP, or you can manually specify SLB, refer to specify purchase SLB and EIP.


Step 4: You’re done. You can now push and pull streams, where HLS streams can be played from Nginx(80), RTMP and HTTP-FLV streams can be played from SRS:

  • The Publish RTMP to RTMP: / / 28.170.32.118 / live/livestream or to origin or to the edge.
  • Play RTMP from RTMP: / / 28.170.32.118 / live/livestream
  • Play the HTTP – FLV from http://28.170.32.118:8080/live/livestream.flv
  • Play HLS from http://28.170.32.118/live/livestream.m3u8


Kubectlget SVC /srs-http-service kubectlget SVC /srs-edge-service kubectlget SVC /srs-edge-service kubectlget SVC /srs-edge-service


Note: If SLB and EIP are created automatically, then the IP of HLS and RTMP/HTTP-FLV are different, you can choose to specify SLB manually, both services can use the same SLB, refer to the specified purchase of SLB and EIP.


Here we choose the stateful cluster Deployment mode or the Deployment mode of the source site. The difference is that each source site needs to create a Deployment and Service. Refer to the Deployment mode of the source site cluster, for example:

  • The 0th source station is named Deployment as SRS-origin-0-deploy and the ClusterIP service as SRS-origin-0.
  • For the first source station, Deployment is named AS SRS-origin-1-deploy and the ClusterIP service is named as SRS-Origin-1.
  • The second source station is named Deployment as SRS-origin-2-DEPLOY and the ClusterIP service as SRS-Origin-2.
  • The source site configuration was changed to fulfill SRS-origin-0 SRS-origin-1 SRS-Origin-2. .
  • Change the edge configuration to Origin SRs-origin-0 srS-origin-1 srS-origin-2. .