If you don’t know what Pipy is, check out GitHub.

Pipy is a lightweight, high-performance, highly stable, and programmable network agent. Pipy’s core framework is developed in C++, and the network IO adopts the ASIO library. Pipy’s executable is only about 5M and its runtime memory footprint is about 10M, so Pipy is a good candidate for Sidecar Proxy. Rego doesn’t work well? OPA Pipy has a built-in PJS script extension that allows Pipy to quickly customize logic and functionality with JS scripts based on specific requirements.

Pipy uses a modular, chained processing architecture that uses sequential modules to process blocks of network data. This simple architecture makes PIPY simple and reliable at the bottom, with the ability to dynamically orchestrate traffic, giving it both simplicity and flexibility. By using the mechanism of REUSE_PORT, which is supported in major Linux and BSD versions, Pipy can be run in multi-process mode, making Pipy suitable not only for Sidecar but also for large-scale traffic handling scenarios. In practice, Pipy is used as a “soft load” when deployed on its own, offering load-balancing capabilities comparable to hardware at low latency, while being flexible and scalable.

After playing with Pipy a few times and exploring how it works, I had more ideas.

  • A preliminary exploration of programmable gateway PIPY
  • Programmable Gateway Pipy’s second shot: programmatically implement Metrics and source code interpretation
  • Programmable gateway PIPY third: event model design

When using OPA, I always felt that Rego was not so smooth, and the idea of using Pipy JS to write rules came into my mind. Let’s try it out today. Pipy is not just an “agent”, there are many scenarios that can be applied to it:

  • The tiny single binary makes Pipy perhaps the best “cloud-native sidecar”
  • Sidecar is not just a proxy, it can be a controller, it can be an arithmetic unit
  • Proxy’s string structure is suitable for various control class operations, such as access control
  • With its extensibility and fast programming capabilities, Pipy JS is well suited to being a “rules engine,” or as it has become popular lately, a “cloud-native rules engine.” Compared to OPA, I think it is well qualified to be a “featherweight rule enforcement engine”.

Right now I’m more likely to define Pipy as a “cloud-native traffic programming framework” where proxies are just the underlying core capabilities, but there’s a lot more you can do with Pipy JS on top of it, and “traffic feeds everything.”

After implementing the trusted mirror repository check using the Open Policy Agent, I wondered if Pipy could do the same, replacing the kernel with Pipy + rules. So most of today’s content is similar to the one above.

Let’s take a look at how Pipy implements Kubernetes’ access controller to check the image.

The environment

Keep using Minikube

minikube start

Create the namespace in which Pipy is deployed

Kubectl create namespace pipy kubens pipy kubectl label ns pipy pipy/webhook=ignore #

The rules

In OPA, the kube-mGMT container monitors the changes in configMap and pushes the Policy to the OPA container of the same POD.

In the case of Pipy for gradients, mount the configMap containing the rules directly into Pipy’s container using mount mode.

In practice, PIPY supports rotating training to check the changes of rules in the control plane and load them in real time. You can also implement the same logic as OPA’s Kube-MGMT.

The Pipy rule that implements the functionality of the previous lecture is as follows:

Cat > pipy-rule.js <<EOF pipy({_repopfix: '192.168.64.1', //192.168.64.1:5000 is a private repository running on my local container. _tagSuffix: ':latest', }) .listen(6443, { tls: { cert: os.readFile('/certs/tls.crt').toString(), key: os.readFile('/certs/tls.key').toString(), }, }) .decodeHttpRequest() .replaceMessage( msg => ( ((req, result, invalids, reason) => ( req = JSON.decode(msg.body), invalids = req.request.object.spec.containers.find(container => ( (! container.image.startsWith(_repoPrefix) ? ( reason = `${container.image} repo not start with ${_repoPrefix}`, console.log(reason), true ) : (false)) || (container.image.endsWith(_tagSuffix) ? ( reason = `${container.image} tag end with ${_tagSuffix}`, console.log(reason), true ) : (false) ))), invalids ! = undefined ? ( result = { "apiVersion": "admission.k8s.io/v1beta1", "kind": "AdmissionReview", "response": { "allowed": false, "uid": req.request.uid, "status": { "reason": reason, }, }, } ) : ( result = { "apiVersion": "admission.k8s.io/v1beta1", "kind": "AdmissionReview", "response": { "allowed": true, "uid": req.request.uid }, } ), console.log(JSON.encode(result)), new Message({ 'status' : 200, 'headers': { 'Content-Type': 'application/json' } }, JSON.encode(result)) ))() ) ) .encodeHttpResponse() EOF

Save the rule in configMap:

kubectl create configmap pipy-rule --from-file=pipy-rule.js

Deploy Pipy on Kubernetes

Kubernetes uses TLS to communicate with the Admission Controller. Configure TLS to create certificate/secret key pairs for Certificate Authority CA and OPA using OpenSSL.

openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -days 100000 -out ca.crt -subj "/CN=admission_ca"

Create TLS secret keys and certificates for OPA:

cat >server.conf <<EOF
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
prompt = no
[req_distinguished_name]
CN = pipy.pipy.svc
[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1 = pipy.pipy.svc
EOF

Pay attention to
CN 和
alt_namesMust match the Pipy Service created later. 

openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -config server.conf
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 100000 -extensions v3_req -extfile server.conf

Create a Secret for OPA that holds TLS credentials:

kubectl create secret tls pipy-server --cert=server.crt --key=server.key

Deploy PIPY as the Admission Controller. To facilitate debugging, we opened the console when we started Pipy.

kind: Service apiVersion: v1 metadata: name: pipy namespace: pipy spec: selector: app: pipy ports: - name: HTTPS Protocol: TCP Port: 443 targetPort: 6443-name: GUI http protocol: TCP port: 6080 targetPort: 6080 --- apiVersion: apps/v1 kind: Deployment metadata: labels: app: pipy namespace: pipy name: pipy spec: replicas: 1 selector: matchLabels: app: pipy template: metadata: labels: app: pipy name: pipy spec: containers: - name: pipy image: pipy:latest imagePullPolicy: IfNotPresent args: -- "pipy" -- "/opt/data/pipy-rule.js" -- "--gui-port=6060" # - "--log-level=debug" ports: -name: gui containerPort: 6060 protocol: TCP - name: http containerPort: 6080 protocol: TCP - name: https containerPort: 6443 protocol: TCP volumeMounts: - readOnly: true mountPath: /certs name: pipy-server - readOnly: false mountPath: /opt/data name: pipy-rule volumes: - name: pipy-server secret: secretName: pipy-server - name: pipy-rule configMap: name: pipy-rule

Access to the exposed console:

kubectl expose deploy pipy --name pipy-node --type NodePort kubectl get svc pipy-port minikube service --url pipy-node -n pipy # Find the console port

Next, generate the manifest that will be used to register Pipy as an access controller.

cat > webhook-configuration.yaml <<EOF
kind: ValidatingWebhookConfiguration
apiVersion: admissionregistration.k8s.io/v1beta1
metadata:
  name: pipy-validating-webhook
webhooks:
  - name: validating-webhook.pipy.flomesh-io.cn
    namespaceSelector:
      matchExpressions:
      - key: pipy/webhook
        operator: NotIn
        values:
        - ignore
    rules:
      - operations: ["CREATE", "UPDATE"]
        apiGroups: ["*"]
        apiVersions: ["*"]
        resources: ["pods"]
    clientConfig:
      caBundle: $(cat ca.crt | base64 | tr -d '\n')
      service:
        namespace: pipy
        name: pipy
EOF

The generated configuration file contains the Base64 encoding of the CA certificate so that a TLS connection can be established between the Kubernetes API server and the OPA.

kubectl apply -f webhook-configuration.yaml

test

pod-bad-repo.yaml:

apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: web-server name: web-server namespace: Default spec: containers: -image: nginx:1.21.1 name: web-server resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {}
Kubectl apply-f pod-bad-repo. Yaml Error from server (nginx:1.21.1 repo not start with 192.168.64.1): error when creating "pod-bad-repo.yaml": admission webhook "validating-webhook.pipy.flomesh-io.cn" denied the request: Nginx :1.21.1 repo not start with 192.168.64.1

pod-bad-tag.yaml

apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: web-server name: web-server namespace: The default spec: containers: - image: 192.168.64.1:5000 / nginx: latest name: web - server resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {}
Kubectl apply -f pod - bad - tag. Yaml Error from the server (192.168.64.1:5000 / nginx: latest tag end with: latest) : error when creating "pod-bad-tag.yaml": admission webhook "validating-webhook.pipy.flomesh-io.cn" denied the request: 192.168.64.1:5000 / nginx: latest tag end with: the latest

pod-ok.yaml

apiVersion: v1 kind: Pod metadata: creationTimestamp: null labels: run: web-server name: web-server namespace: The default spec: containers: - image: 192.168.64.1:5000 / nginx: 1.21.1 name: web - server resources: {} dnsPolicy: ClusterFirst restartPolicy: Always status: {}
kubectl apply -f pod-ok.yaml
pod/web-server created

conclusion

OPA is good for everything, the only drawback is that the introduction of the Rego language raises the bar for use. Pipy’s rules are written by JavaScript, and the front-end students can also write the rules. Full substitution may be a bit of an exaggeration, but it can be used to replace OPA in some scenarios.

At this point, you’ll notice that with rules and powerful filters (I like to call them hooks now), Pipy is very playable.

For example, OPA: Kubernetes’ Top 5 Access Control Strategies, such as… . Bold imagination.

Want to write a series called “How to Play Pipy Bad”?

The article is uniformly published in the public cloud native refers to the north