如何从集群外部访问我的Cassandra / Kubernetes集群?

Manu Chadha

我已经开始使用Cass-Operator,并且安装过程像个魅力!https://github.com/datastax/cass-operator

我有一个问题。我的集群已启动并在GCP上运行。但是,如何从笔记本电脑(基本上是从外部)访问它?抱歉,我是Kubernetes的新手,所以我不知道如何从外部访问集群?

我可以看到节点在GCP仪表板上。我可以从笔记本电脑ping节点的外部IP,但是当我运行时cqlsh external_ip 9042,连接失败。

如何将K8s / Cassandra群集连接到外部工作,以便我的Web应用程序可以访问它?

我想要:

  1. 有一个URL,以便我的Web应用程序使用该URL而不是IP地址来连接到cassandra / K8s集群。因此,我需要一个DNS。K8S是否默认提供它?会是网址吗?在某些节点中为我管理dns映射的K8会重启吗?
  2. 我的Web应用程序应该能够在9042上访问Cassandra。似乎已经为http / https完成了负载平衡。该Cassandra应用程序不是http / https请求。所以我不需要80或443端口

我读过一些有关服务,负载平衡器和入口的教程。但是我无法开始。

我创建了这样的服务

kind: Service
apiVersion: v1
metadata:
  name: cass-operator-service
spec:
  type: LoadBalancer
  ports:
    - port: 9042
  selector:
    name: cass-operator

然后创建服务- kubectl apply -f ./cass-operator-service.yaml

我检查了服务是否是使用创建的kubectl get svc并获得了输出

NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)          AGE 
cass-operator-service   LoadBalancer   10.51.249.224   34.91.214.233   9042:30136/TCP   4m17s 
kubernetes              ClusterIP      10.51.240.1     <none>          443/TCP          10h. 

但是当我运行时cqlsh 34.91.214.233 9042,连接失败

似乎对端口9042的请求将转发到30136。但是应该将它们转发到9042,因为那是Pod中的Cassandra映像侦听传入请求的地方

更新

尝试了targetPort但仍然没有运气

manuchadha25@cloudshell:~ (copper-frame-262317)$ cat cass-operator-service.yaml
kind: Service
apiVersion: v1
metadata:
  name: cass-operator-service
spec:
  type: LoadBalancer
  ports:
    - port: 9042
      targetPort: 9042
  selector:
    name: cass-operator
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.51.240.1   <none>        443/TCP   11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl apply -f ./cass-operator-service.yaml
service/cass-operator-service created
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
cass-operator-service   LoadBalancer   10.51.255.184   <pending>     9042:30024/TCP   12s
kubernetes              ClusterIP      10.51.240.1     <none>        443/TCP          11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
cass-operator-service   LoadBalancer   10.51.255.184   <pending>     9042:30024/TCP   37s
kubernetes              ClusterIP      10.51.240.1     <none>        443/TCP          11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get service
NAME                    TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)          AGE
cass-operator-service   LoadBalancer   10.51.255.184   34.91.214.233   9042:30024/TCP   67s
kubernetes              ClusterIP      10.51.240.1     <none>          443/TCP          11h
manuchadha25@cloudshell:~ (copper-frame-262317)$ ping 34.91.214.233
PING 34.91.214.233 (34.91.214.233) 56(84) bytes of data.
64 bytes from 34.91.214.233: icmp_seq=1 ttl=109 time=7.89 ms

在此处输入图片说明

查询所有名称空间将显示以下内容

在此处输入图片说明

但是使用命名空间cass-operator查询pod会返回空结果

manuchadha25@cloudshell:~ (copper-frame-262317)$ kubectl get pods -l name=cass-operator
No resources found in default namespace.
请问ROF
  • 由于您是Kubernetes的新手,因此您可能不熟悉StatefulSets

StatefulSet是用于管理有状态应用程序的工作负载API对象。

管理一组的部署和缩放窗格并提供有关的排序和唯一性保证这些荚。

Like a Deployment, a StatefulSet manages Pods that are based on an identical container spec. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.


How do I go about connecting the K8s/Cassandra cluster to outside work so that my web application can access it?

  • I found out that datastax/cass-operator is still developing their documentation, I found this document that is not merged to master yet, but it explains very well about how to connect to Cassandra, I strongly recommend reading.
  • There are several open issues for documenting methods for connection from outside the cluster.

I followed the guide in https://github.com/datastax/cass-operator to deploy the cass-operator + Cassandra Datacenter Example as from your images I believe you followed as well:

$ kubectl create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/docs/user/cass-operator-manifests-v1.15.yaml
namespace/cass-operator created
serviceaccount/cass-operator created
secret/cass-operator-webhook-config created
customresourcedefinition.apiextensions.k8s.io/cassandradatacenters.cassandra.datastax.com created
clusterrole.rbac.authorization.k8s.io/cass-operator-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/cass-operator created
role.rbac.authorization.k8s.io/cass-operator created
rolebinding.rbac.authorization.k8s.io/cass-operator created
service/cassandradatacenter-webhook-service created
deployment.apps/cass-operator created
validatingwebhookconfiguration.admissionregistration.k8s.io/cassandradatacenter-webhook-registration created

$ kubectl create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/operator/k8s-flavors/gke/storage.yaml
storageclass.storage.k8s.io/server-storage created

$ kubectl -n cass-operator create -f https://raw.githubusercontent.com/datastax/cass-operator/v1.2.0/operator/example-cassdc-yaml/cassandra-3.11.6/example-cassdc-minimal.yaml
cassandradatacenter.cassandra.datastax.com/dc1 created

$ kubectl get all -n cass-operator
NAME                                READY   STATUS    RESTARTS   AGE
pod/cass-operator-78c6469c6-6qhsb   1/1     Running   0          139m
pod/cluster1-dc1-default-sts-0      2/2     Running   0          138m
pod/cluster1-dc1-default-sts-1      2/2     Running   0          138m
pod/cluster1-dc1-default-sts-2      2/2     Running   0          138m

NAME                                          TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)             AGE
service/cass-operator-metrics                 ClusterIP      10.21.5.65    <none>         8383/TCP,8686/TCP   138m
service/cassandradatacenter-webhook-service   ClusterIP      10.21.0.89    <none>         443/TCP             139m
service/cluster1-dc1-all-pods-service         ClusterIP      None          <none>         <none>              138m
service/cluster1-dc1-service                  ClusterIP      None          <none>         9042/TCP,8080/TCP   138m
service/cluster1-seed-service                 ClusterIP      None          <none>         <none>              138m

NAME                            READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/cass-operator   1/1     1            1           139m

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/cass-operator-78c6469c6   1         1         1       139m

NAME                                        READY   AGE
statefulset.apps/cluster1-dc1-default-sts   3/3     138m

$ CASS_USER=$(kubectl -n cass-operator get secret cluster1-superuser -o json | jq -r '.data.username' | base64 --decode)
$ CASS_PASS=$(kubectl -n cass-operator get secret cluster1-superuser -o json | jq -r '.data.password' | base64 --decode)

$ echo $CASS_USER
cluster1-superuser

$ echo $CASS_PASS
_5ROwp851l0E_2CGuN_n753E-zvEmo5oy31i6C0DBcyIwH5vFjB8_g
  • From the kubectl get all command above we can see there is an statefulset called statefulset.apps/cluster1-dc1-default-sts which controls the cassandra pods.
  • In order to create a LoadBalancer service that makes available all the pods managed by this statefulset we need to use the same labels assigned to them:
$ kubectl describe statefulset cluster1-dc1-default-sts -n cass-operator
Name:               cluster1-dc1-default-sts
Namespace:          cass-operator
CreationTimestamp:  Tue, 30 Jun 2020 12:24:34 +0200
Selector:           cassandra.datastax.com/cluster=cluster1,cassandra.datastax.com/datacenter=dc1,cassandra.datastax.com/rack=default
Labels:             app.kubernetes.io/managed-by=cass-operator
                    cassandra.datastax.com/cluster=cluster1
                    cassandra.datastax.com/datacenter=dc1
                    cassandra.datastax.com/rack=default
  • Now let's create the LoadBalancer service yaml and use the labels as selectors for the service:
apiVersion: v1
kind: Service
metadata:
  name: cassandra-loadbalancer
  namespace: cass-operator
  labels:
    cassandra.datastax.com/cluster: cluster1
    cassandra.datastax.com/datacenter: dc1
    cassandra.datastax.com/rack: default
spec:
  type: LoadBalancer
  ports:
  - port: 9042
    protocol: TCP
  selector:
    cassandra.datastax.com/cluster: cluster1
    cassandra.datastax.com/datacenter: dc1
    cassandra.datastax.com/rack: default

"My web application should be able to reach Cassandra on 9042. It seems load balancing is done for http/https. The Cassandra application is not a http/https request. So I don't need port 80 or 443."

  • When you create a Service of type LoadBalancer, a Google Cloud controller wakes up and configures a network load balancer in your project. The load balancer has a stable IP address that is accessible from outside of your project.

  • The network load balancer supports any and all ports. You can use Network Load Balancing to load balance TCP and UDP traffic. Because the load balancer is a pass-through load balancer, your backends terminate the load-balanced TCP connection or UDP packets themselves.

  • Now let's apply the yaml and note the Endpoint IPs of the pods being listed:

$ kubectl apply -f cassandra-loadbalancer.yaml 
service/cassandra-loadbalancer created

$ kubectl get service cassandra-loadbalancer -n cass-operator 
NAME                     TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)          AGE
cassandra-loadbalancer   LoadBalancer   10.21.4.253   146.148.89.7   9042:30786/TCP   5m13s

$ kubectl describe svc cassandra-loadbalancer -n cass-operator
Name:                     cassandra-loadbalancer
Namespace:                cass-operator
Labels:                   cassandra.datastax.com/cluster=cluster1
                          cassandra.datastax.com/datacenter=dc1
                          cassandra.datastax.com/rack=default
Annotations:              Selector:  cassandra.datastax.com/cluster=cluster1,cassandra.datastax.com/datacenter=dc1,cassandra.datastax.com/rack=default
Type:                     LoadBalancer
IP:                       10.21.4.253
LoadBalancer Ingress:     146.148.89.7
Port:                     <unset>  9042/TCP
TargetPort:               9042/TCP
NodePort:                 <unset>  30786/TCP
Endpoints:                10.24.0.7:9042,10.24.2.7:9042,10.24.3.9:9042
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>
  • To test it, I'll use my cloud shell with a cassandra container to emulate your notebook using the LoadBalancer IP provided above:
$ docker run -it cassandra /bin/sh

# cqlsh -u cluster1-superuser -p _5ROwp851l0E_2CGuN_n753E-zvEmo5oy31i6C0DBcyIwH5vFjB8_g 146.148.89.7 9042                

Connected to cluster1 at 146.148.89.7:9042.
[cqlsh 5.0.1 | Cassandra 3.11.6 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cluster1-superuser@cqlsh> select * from system.peers;

 peer      | data_center | host_id                              | preferred_ip | rack    | release_version | rpc_address | schema_version                       | tokens
-----------+-------------+--------------------------------------+--------------+---------+-----------------+-------------+--------------------------------------+--------------------------
 10.24.3.9 |         dc1 | bcec6c12-49a1-41d5-be58-5150e99f5dfb |         null | default |          3.11.6 |   10.24.3.9 | e84b6a60-24cf-30ca-9b58-452d92911703 |  {'2248175870989649036'}
 10.24.0.7 |         dc1 | 68409f08-9d6e-4e40-91ff-f43581c8b6f3 |         null | default |          3.11.6 |   10.24.0.7 | e84b6a60-24cf-30ca-9b58-452d92911703 | {'-1105923522927946373'}

(2 rows)

"have a url so that my web application uses that URL to connect to the cassandra/K8s cluster instead of IP address. So I need a dns. Does it come by default in K8S? Would would be the url? Would K8s managing the dns mapping for me in some nodes get restarted?"

  • 关于cassandra-operator的文档也有一个关于Ingress的部分,我也建议阅读。
  • Kubernetes没有默认的DNS名称。
  • 您将必须注册一个域,将DNS指向负载均衡器的IP,这样它将解析网络LoadBalancer的IP。
  • 网络负载平衡器已绑定到静态公共IP,Kubernetes节点中的任何更改都不会导致服务不可用。

如有任何疑问,请在评论中告诉我。

本文收集自互联网,转载请注明来源。

如有侵权,请联系 [email protected] 删除。

编辑于
0

我来说两句

0 条评论
登录 后参与评论

相关文章