this post is WIP
k8sServiceTunnel is a collection of utilities and containers i created to scratch my own itch. As i operate both a k8s cluster and a few independent systems i often need a way to access services hosted on my k8s. These systems are often instances in various cloud providers which i don't really trust and being test systems they ofter run insecure code. So sharing k8s secrets with them is ..ehm.. I also don't feel comfortable to expose the service on my k8s cluster.
There are some solutions i could think of but none matched my needs. What i needed was:
- A way to access these services without disclosing any k8s credentials to those external hosts.
- A way for the services to be persistent and self healing in case the tunnel between them fails.
- As my k8s nodes have both internal and external IPs and specifying interfaces on nodePorts is not a thing (and probably will never be) i was not comfortable on having exposed WAN ports.
- Firewalling WAN on the instances was not possible as well (long story for another time).
What if we could easily establish an ssh tunnel from a pod in k8s to the external host and using remote port forwarding expose the service there? Furthermore it would be nice if the ssh private key remained in k8s and the external host thats receiving the service endpoint only has a public key.
Thats the concept behind k8sServiceTunnel!
Some of the benefits of k8sServiceTunnel:
- Host can be untrusted as no k8s credentials need to be shared and connection is controlled from the client side (aka the k8s pod). No secrets are found in the host receiving the service.
- a service with a dynamic DNS can be shared/forwarded and will always be up to date as pods are to die and be replaced.
- Can work over wan/lan and is fully configurable using env vars in the containers.
Lets consider an example scenario:
- We have an application running in an externel host that needs access to a percona xtradb mysql service running as pods in k8s.
so as depicted above we have:
(A) An application on an external host that needs access to. (D) a MySQL service running in k8s. (B) k8sServiceTunnel server component running on the remote host (C) k8sServiceTunnel client component running in a k8s pod.
How to run
First, generate an rsa key pair. Store the private part in a k8s secret and set the public base64 encoded for the server. In the external host start Tunnel SSH Server docker using the base64 encoded public part of the generated key as environmental variable.
After that its type for the k8s pod, set the rsa private part in the secret and apply all yamls (WIP, coming soon ...)
After a while we have the client running in k8s:
As seen the client is now exposing the mysql service as a remote port forwarding at port 8999 in the server (the docker container running in the external host).
Let's try connecting to mysql from the remote host: