NuSTER can also be used as an HTTP cache server such as Varnish or Nginx to cache dynamic or static HTTP resources.

• All features of HAProxy (HTTPS, HTTP/2, ACL, etc)
• Very fast
• Powerful dynamic caching
  • Based on HTTP method, URI, path, Query, header, cookies, etc
  • Based on HTTP request or response contents, etc
  • Based on environment variables, server state, etc
  • Based on SSL version, SNI, etc
  • Based on connection rate, number, byte, etc
• Cache management
• The cache to clear
• Cache statistics
• Cache lifetime

Very fast, 3 times as fast as Nginx in single-process mode, 2 times as fast as Nginx in multi-process mode, and 3 times as fast as Varnish.

See the benchmark

For production environments, Download the latest stable version from Download and git Clone for other environments.

Prepare a configuration file: nuster.cfg

global
    nuster cache on data-size 100m uri /_nuster
    nuster nosql on data-size 200m
defaults
    mode http
frontend fe
    bind *:8080
    #bind *:4433 ssl crt example.com.pem alpn h2,http/1.1
    use_backend be2 if { path_beg /_kv/ }
    default_backend be1
backend be1
    nuster cache on
    nuster rule img ttl 1d if { path_beg /img/ }
    nuster rule api ttl 30s if { path /api/some/api }
    server s1 127.0.0.1:8081
    server s2 127.0.0.1:8082
backend be2
    nuster nosql on
    nuster rule r1 ttl 3600
Copy the code

Nuster listens on port 8080 and accepts HTTP requests. Requests starting with /_kv/ are allocated to backend BE2. HTTP POST, GET, or DELETE can be sent to /_kv or any_key to add, obtain, or DELETE keys or values. Other requests are allocated to Backend be1 and are forwarded to servers S1 or S2. The /img/* request is cached for 1 day, and/API /some/ API is cached for 30 seconds.

docker pull nuster/nuster
docker run -d -v /path/to/nuster.cfg:/etc/nuster/nuster.cfg:ro -p 8080:8080 nuster/nuster
Copy the code

The configuration file has four basic sections: Global, defaults, frontend, and Backend.

• global
  • Define global directives
  • You need to definenuster cache on or nuster nosql onOtherwise, cache and nosql cannot be used
• defaults
  • definefrontend.backendDefault argument to
  • Can be found infrontend or backendSection redefinition
• frontend
  • Define listening ports and other user-oriented Settings
• bankend
  • Define backend servers and so on
  • You need to set upnuster cache on or nuster nosql onOtherwise, Backend does not have noSQL or noSQL functions
  • You need to set upnuster rule

Can define multiple frontend or backend. If defines nuster cache | no off or no nuster cache | no on | off, nuster is a HAProxy.

For details, see the HAProxy documentation under /doc or online

frontend web-lb
   bind *:80
   #bind *:443 ssl crt XXX.pem
   mode http
   default_backend apps
backend apps
   balance roundrobin
   mode http
   server s1 10.0.0.101:8080
   server s2 10.0.0.102:8080
   server s3 10.0.0.103:8080
   #server s4 10.0.0.101:8443 ssl verify none
Copy the code

global
    nuster nosql on data-size 200m
frontend fe
    bind *:8080
    default_backend be
backend be
    nuster nosql on
    nuster rule r1 ttl 3600
Copy the code

syntax:

nuster cache on|off [data-size size] [dict-size size] [purge-method method] [uri uri]

nuster nosql on|off [data-size size] [dict-size size]

default: none

context: global

Controls whether cache or noSQL is enabled.

A data-size + dict-size shared memory is allocated to store HTTP headers, data, keys, etc. Temporary data is allocated from the system memory pool. If there is not enough memory, new requests are not cached until memory is freed.

Determines the size of the Hash table.

You can use m, m, G, and g. The default is 1MB, which is also the minimum.

This determines the size of the hash table buckets, not the key, which is stored in shared memory.

Dict-size is different from the number of keys. Even if the number of keys exceeds dict-size, new keys can be added as long as there is space in the entire shared memory.

However, performance may deteriorate if the number of keys exceeds dict-size(bucket number). Dict-size can be set to the maximum number of keys multiplied by 8.

Dict-size will be removed in future versions and will scale automatically like the first version

Define and enable the Cache Manager/Stats API

nuster cache on uri /_my/_unique/_/_cache/_uri

Cache Manager/STATS is off by default. If yes, access control (SEE FAQ) is enabled.

For details, see Cache Management and Cache Statistics.

syntax: nuster rule name [key KEY] [ttl TTL] [code CODE] [if|unless condition]

default: none

context: backend, listen

To define the effective conditions of cache/ NOSQL, you must define at least one rule.

nuster cache on

# cache request `/asdf` for 30 seconds
nuster rule asdf ttl 30 if { path /asdf }

# cache if the request path begins with /img/
nuster rule img if { path_beg /img/ }

# cache if the response header `cache` is `yes`
acl resHdrCache res.hdr(cache) yes
nuster rule r1 if resHdrCache
Copy the code

Multiple rules can be defined and matched in the order they are defined.

acl pathA path /a.html
nuster cache on
nuster rule all ttl 3600
nuster rule path01 ttl 60 if pathA
Copy the code

Rule Path01 will never be matched.

Defines the cache/ noSQL key, supplemented by the following keywords. composition

• method: http method, GET/POST…
• scheme: http or https
• host: the host in the request
• uri: first slash to end of the url
• path: the URL path of the request
• delimiter: ‘? ‘ if query exists otherwise empty
• query: the whole query string of the request
• header_NAME: the value of header NAME
• cookie_NAME: the value of cookie NAME
• param_NAME: the value of query NAME
• body: the body of the request

The default CACHE key is method.scheme.host.uri, and the default NoSQL key is get.scheme.host. uri.

Example

GET http://www.example.com/q?name=X&type=Y HTTP header: GET /q? Name =X&type=Y HTTP/1.1 Host: www.example.com ASDF: Z Cookie: logged_in=yes; user=nuster;Copy the code

Generation:

• method: GET
• scheme: http
• host: www.example.com
• uri: /q? name=X&type=Y
• path: /q
• delimiter: ?
• query: name=X&type=Y
• header_ASDF: Z
• cookie_user: nuster
• param_type: Y
• body: (empty)

The default GET.http.www.example.com./q?name=X&type=Y. key, And the key method. The scheme. Host. Path. Header_ASDF. Cookie_user. Param_type GET.http.www.example.com./q.Z.nuster.Y.. is generated

Requests with the same key are directly returned to the cache.

By default, only 200 responses are cached. If you need to cache others, you can add them. All will cache any status code.

Cache-rule only200 cache-rule 200and404 code 200,404 cache-rule all code allCopy the code

NuSTER can also be used as an HTTP cache server such as Varnish or Nginx to cache dynamic or static HTTP resources. HAProxy provides SSL, HTTP, HTTP2, rewrite redirection, add, delete, change Header, and so on, as well as the following functions.

A rule can be dynamically enabled or disabled through the Manager URI. The disabled rule will not be matched.

headers

All rules with the same name are enabled or disabled.

Examples

• Close the rule r1

  The curl -x POST "name: r1" - H - H "state: disable" http://127.0.0.1/nuster/cache

• None Example Disable all rules of Backend APP1b

  The curl -x POST "name: app1b" - H - H "state: disable" http://127.0.0.1/nuster/cache

• Enable all rules

  The curl -x POST "name: *" - H - H "state: enable" http://127.0.0.1/nuster/cache

Set both state and TTL

Curl -x POST "name: r1" - H - H "TTL: 0 -" H "state: enabled" http://127.0.0.1/nuster/cacheCopy the code

The curl - XPURGE https://127.0.0.1/imgs/test.jpg

Generate the key get.scheme.host. uri and delete that key.

The default key contains Host. If http://example.com/test is used in the cache, the Host header is required:

Curl -xpurge -h "Host: example.com" http://127.0.0.1/test

Delete all caches belonging to this host by wearing the X-Hostheader.

headers

Examples

The curl -x PURGE -h "X - host: 127.0.0.1:8080" http://127.0.0.1/nuster/cacheCopy the code

It can also be deleted by re, and all caches that match the re are deleted.

headers

Examples

Curl -x PURGE -h "regex: ^ / imgs /. * \. JPG $" http://127.0.0.1/nuster/cache # delete all caches which path starts with/imgs and ends with the. JPG and Belongs to 127.0.0.1:8080 curl -x PURGE -h "regex: ^ / imgs /. * \. JPG $" -h" 127.0.0.1:8080 "http://127.0.0.1/nuster/cacheCopy the code

PURGE Considerations

1. Enabling Access Control

2. If there are multiple headers, process them in the order name, path & host, path, regex & Host, regex, and host

   curl -XPURGE -H "name: rule1" -H "path: /imgs/a.jpg": purge by name

3. If there are duplicate headers, process the first one

   curl -XPURGE -H "name: rule1" -H "name: rule2": purge by rule1

4. The regex isn’t glob

   For example, the.jpg file under /imgs is ^/imgs/.*\.jpg$instead of /imgs/*.jpg

nuster cache on uri /nuster/cache
Copy the code

• Used_mem: memory used by the HTTP cache, not including overhead
• Req_total: indicates the total number of requests from all backend systems with cache enabled, excluding requests from backend systems that do not have cache
• Req_hit: indicates the number of cache hits
• Req_fetch: Fetch the quantity from the back end
• Req_abort: Abort request

The curl -v http://127.0.0.1:8080/key1

Check status code.

• 200 OK
  • POST/GET: success
  • Always DELETE:
• 400 Bad request
  • A null value
  • Incorrect ACL, rules, etc
• 404 Not Found
  • POST: Rule tests fail
  • GET: not found
• 405 Method Not Allowed
  • The methods of the other
• 500 Internal Server Error
  • Unknown error occurred
• 507 Insufficient Storage
  • More than data – the size

Curl -v - X POST - d "333" H "userId: 1000" curl http://127.0.0.1:8080/mypoint - v - X POST - d "555" - H "userId: 1001 "http://127.0.0.1:8080/mypoint curl -v - X POST - d" userA data "-- cookies" sessionId: Ijsf023xe "http://127.0.0.1:8080/mydata curl -v - X POST - d" userB data "-- cookies" sessionId: Rosre329x "http://127.0.0.1:8080/mydataCopy the code

Support any CLIENT that supports HTTP, libraries: curl, Postman, Python requests, go net/ HTTP, etc.

Add debug in global, or start haProxy with -d

Cache-related debugging information starts with [CACHE]

similar

Acl network_allowed SRC 127.0.0.1 ACL purge_method method PURGE http-request deny if purge_method! network_allowedCopy the code

global nuster cache on data-size 100m nuster nosql on data-size 100m #daemon ## to debug cache #debug defaults retries 3  option redispatch timeout client 30s timeout connect 30s timeout server 30s frontend web1 bind *:8080 mode http acl pathPost path /search use_backend app1a if pathPost default_backend app1b backend app1a balance roundrobin # mode must be http mode http # http-buffer-request must be enabled to cache post request option http-buffer-request acl pathPost path /search # enable cache for this proxy nuster cache # cache /search for 120 seconds. Only works when POST/PUT nuster Rule rpost key method. The scheme. Host. Uri. The body TTL 120 if pathPost server s1 10.0.0.10:8080 backend app1b balance roundrobin mode http nuster cache on # cache /a.jpg, not expire acl pathA path /a.jpg nuster rule r1 ttl 0 if pathA # cache /mypage, key contains cookie[userId], so it will be cached per user acl pathB path /mypage nuster rule r2 key method.scheme.host.path.delimiter.query.cookie_userId ttl 60 if pathB # cache /a.html if response's header[cache] is yes  http-request set-var(txn.pathC) path acl pathC var(txn.pathC) -m str /a.html acl resHdrCache1 res.hdr(cache) yes nuster  rule r3 if pathC resHdrCache1 # cache /heavy for 100 seconds if be_conn greater than 10 acl heavypage path /heavy acl tooFast be_conn ge 100 nuster rule heavy ttl 100 if heavypage tooFast # cache all if response's header[asdf] is fdsa acl ResHdrCache2 res. HDR (ASDF) fdsa nuster rule resCache TTL 0 if resHdrCache1 server s1 10.0.0.10:8080 frontend web2 bind *:8081 mode http default_backend app2 backend app2 balance roundrobin mode http # disable cache on this proxy nuster Cache off nuster rule All Server S2 10.0.0.11:8080 listen web3 Bind *:8082 mode HTTP nuster cache nuster rule everything Server S3 10.0.0.12.8080 frontend noSQL_fe bind *:9090 default_backend nosql_be backend nosql_be nuster noSQL on nuster  rule r1 ttl 3600Copy the code

• Join the development
• Give feedback
• Report issues
• Send pull requests
• Spread nuster

All rights reserved.

Licensed under GPL, the same as HAProxy

HAProxy and other sources license notices: see relevant individual files.