068- Write a tool to synchronize eureka/ NacOS instances to Apisix/Kong using GO

Run through binary

Download the latest binary for the corresponding system from Releases

discovery-syncer-windows-amd64.exe --help

usage: discovery-syncer-windows-amd64.exe [<flags>]

Flags:
  -h, --help  Show context-sensitive help (also try --help-long and --help-man).
  -p, --web.listen-address=": 8080"
              The address to listen on for web interface.
  -c, --config.file="config.yml"
              Path to configuration file.
Copy the code

The configuration file

Whether to enable pprof
# via http://ip:port/debug/pprof/ access
enable-pprof: false
logger:
    level: debug # debug,info,error
    logger: console # console vs file
    log-file: syncer.log # The file name of the logger output, does not exist automatically
    date-slice: y # Cut the document by date, support "y" (year), "m" (month), "d" (day), "h" (hour), default "y".

# registry,map form
discovery-servers:
    # nacos1 is the name of the registry, which can be defined arbitrarily but cannot be repeated
    nacos1:
        # type, currently only nacOS and Eureka are supported
        type: nacos
        The default weight to add if the registry does not return the weight
        weight: 100
        # registry URL prefix
        prefix: /nacos/v1/
        The address of the registry should not end with /
        host: "http://nacos-server:8858"
    eureka1:
        type: eureka
        weight: 100
        prefix: /eureka/
        For basic authentication, you can write this
        host: "http://admin:admin@eureka-server:8761"

# gateway,map form
gateway-servers:
    The gateway name can be written arbitrarily, but cannot be repeated
    apisix1:
        Gateway type, currently supports Apisix and Kong
        type: apisix
        # host = host
        admin-url: http://apisix-server:9080
        # Manage uri prefix
        prefix: /apisix/admin/
        # special extension parameters are added as key:value in config
        config:
            X-API-KEY: xxxxx
    kong1:
        type: kong
        admin-url: http://kong-server:8001
        prefix: /upstreams/

# Synchronize tasks in list form
targets:
    # Note the same registry, but with multiple tenants (nacos-like namespaces), there is no need to create multiple identical registries
    Create targets and change the config extension parameter
    # First task
    # Registry, source
    -   discovery: nacos1
        # gateway, target
        gateway: apisix1
        # Enable/disable
        enabled: false
        Support the expression, # pull interval, for more information on https://github.com/robfig/cron
        fetch-interval: "@every 10s"
        The default is to pull all the services in the registry that can be pulled down, and exclude those that are not needed
        exclude-service: [ 'ex*'.'test' ]
        Prefixes the name of upstream synchronized to the gateway for ease of administration
        upstream-prefix: nacos1
        # For health check, if the number of seconds exceeds the limit, the state is considered disconnected. The default value is 10 seconds
        maximum-interval-sec: 20
        # extension parameters
        config:
            # nacos groupName
            groupName: DEFAULT_GROUP
            # nacos namespace
            namespaceId: test
            Create a default template for upstream to apisix. Search golang text/template for supported template syntax
            template: | { "id": "syncer-{{.Name}}", "timeout": { "connect": 30, "send": 30, "read": 30 }, "name": "{{.Name}}", "nodes": {{.Nodes}}, "type":"roundrobin", "desc": "auto sync by https://github.com/anjia0532/discovery-syncer" }
    -   discovery: eureka1
        gateway: kong1
        enabled: false
        fetch-interval: "@every 5s"
        maximum-interval-sec: 10
        config:
            template: | { "name": "{{.Name}}", "algorithm": "round-robin", "hash_on": "none", "hash_fallback": "none", "hash_on_cookie_path": "/", "slots": 10000, "healthchecks": { "passive": { "healthy": { "http_statuses": [200, 201, 202, 203, 204, 205, 206, 207, 208, 226, 300, 301, 302, 303, 304, 305, 306, 307, 308], "successes": 0 }, "type": "http", "unhealthy": { "http_statuses": [429, 500, 503], "timeouts": 0, "http_failures": 0, "tcp_failures": 0 } }, "active": { "timeout": 1, "https_sni": "example.com", "http_path": "/", "concurrency": 10, "https_verify_certificate": true, "type": "http", "healthy": { "http_statuses": [200, 302], "successes": 0, "interval": 0 }, "unhealthy": { "http_statuses": [429, 404, 500, 501, 502, 503, 504, 505], "timeouts": 0, "http_failures": 0, "interval": 0, "tcp_failures": 0 } }, "threshold": 0 }, "tags": ["discovery-syncer-auto"] }Copy the code

To optimize PM

1. The current synchronization task is serial, if the amount to be synchronized is large, or the synchronization time window is set very small, it will lead to extrusion
2. Custom synchronization plug-ins are not supported, which is not conducive to self-expansion
3. The synchronization mechanism is currently based on timed polling, which is inefficient and needs to be optimized, such as adding cache switches, not pulling/changing upstream information on downstream gateways if there is no difference in the ratio between upstream registries and cache, or checking whether the registry supports active notification of changes.

The resources

• My blog
• I’m the nuggets