1 什么是K8ssandra
Cassandra是一款非常优秀的开源的分布式NoSQL数据库，被许多优秀的大公司采用，具有高可用、弹性扩展、性能好等特点。
正应Cassandra的优势，我们经常需要在云上服务使用，则需要部署Cassandra到K8s上，这就有了K8ssandra。K8ssandra不仅帮助我们可以快速可靠地在Kubernetes上部署Cassandra，同时提供了许多组件，如监控、备份、同步、访问等。而这些都是一个Production-Ready的产品不可或缺的。
K8ssandra的组件架构图如下：
Cass-operator：保证整个集群正常运行；
Reaper：保证一致性的同步工具；
Medusa：数据备份工具，支持S3、GCP Cloud Storage等；
Stargate：对数据访问提供API；
Prometheus+Grafana：云原生的常用监控组件。
2 安装K8ssandra
2.1 安装Kubenetes
如何在Ubuntu上搭建Kubernetes，在文章《VirtualBox+Ubuntu16搭建Kubernetes集群的实现》已经有详细的介绍，这里不再赘述。为了更好的兼容性，我们指定了Kubernetes的版本，命令如下：
minikube start --driver=none --kubernetes-version=v1.19.13
|
因为要用到PVC，我们创建一个StorageClass：
$ kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
|
2.2 安装helm3
我们需要使用Helm来部署K8ssandra，下载Helm3如下：
# 下载安装包
curl -LO https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
# 解压
tar -zxvf helm-v3.7.0-linux-amd64.tar.gz
# 移动到bin目录
mv linux-amd64/helm /usr/local/bin/helm
|
添加Helm的仓库：
helm repo add k8ssandra https://helm.k8ssandra.io/stable
$ helm repo list
NAME URL
k8ssandra https://helm.k8ssandra.io/stable
traefik https://helm.traefik.io/traefik
|
查找一下K8ssandra相关的包：
$ helm search repo k8ssandra
NAME CHART VERSION APP VERSION DESCRIPTION
k8ssandra/k8ssandra 1.3.1 Provisions and configures an instance of the en...
k8ssandra/k8ssandra-common 0.28.4 Helper library containing functions used by man...
k8ssandra/k8ssandra-operator 0.31.0 1.0.0 Kubernetes operator which handles the provision...
k8ssandra/backup 0.26.0 Creates a CassandraBackup custom resource insta...
k8ssandra/cass-operator 0.31.0 1.8.0 Kubernetes operator which handles the provision...
k8ssandra/medusa-operator 0.30.1 0.1.0 Installs and configures the Medusa Operator for...
k8ssandra/reaper-operator 0.32.1 0.1.0 Configures and installs the Reaper Operator for...
k8ssandra/restore 0.27.1 Creates a CassandraRestore custom resource inst...
|
我们安装k8ssandra/k8ssandra就可以了。
2.3 用Helm安装K8ssandra
Helm是一个Chart+Value的管理方式，我们准备一个yaml文件（k8ssandra-values.yaml）来放一些变量：
cassandra:
version: "4.0.0"
cassandraLibDirVolume:
storageClass: local-path
size: 5Gi
allowMultipleNodesPerWorker: true
heap:
size: 1G
newGenSize: 1G
resources:
requests:
cpu: 1000m
memory: 2Gi
limits:
cpu: 1000m
memory: 2Gi
datacenters:
- name: dc1
size: 1
racks:
- name: default
kube-prometheus-stack:
grafana:
adminUser: admin
adminPassword: admin123
stargate:
enabled: true
replicas: 1
heapMB: 256
cpuReqMillicores: 200
cpuLimMillicores: 1000
|
安装K8ssandra：
$ helm install -f k8ssandra-values.yaml k8ssandra k8ssandra/k8ssandra
NAME: k8ssandra
LAST DEPLOYED: Thu Sep 30 21:20:49 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
|
做一些必要的检查如下：
$ helm list
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
k8ssandra default 1 2021-09-30 21:20:49.409672869 +0800 CST deployed k8ssandra-1.3.1
$ kubectl get cassandradatacenters
NAME AGE
dc1 4m34s
$ kubectl describe CassandraDataCenter dc1 | grep "Cassandra Operator Progress:"
Cassandra Operator Progress: Ready
|
检查一下Pod和Service：
获取K8ssandra超级用户的用户名和密码
$ kubectl get secret k8ssandra-superuser -o jsonpath="{.data.username}" | base64 --decode ; echo
k8ssandra-superuser
$ kubectl get secret k8ssandra-superuser -o jsonpath="{.data.password}" | base64 --decode ; echo
TNE5xOk45C1aQsj29qxw
|
2.4 增加节点
我们为了高可用和容量，创建更多的Cassandra Node，直接修改k8ssandra-values.yaml如下：
cassandra:
version: "4.0.0"
cassandraLibDirVolume:
storageClass: local-path
size: 5Gi
allowMultipleNodesPerWorker: true
heap:
size: 1G
newGenSize: 1G
resources:
requests:
cpu: 1000m
memory: 2Gi
limits:
cpu: 1000m
memory: 2Gi
datacenters:
- name: dc1
size: 3
racks:
- name: racks1
- name: racks2
- name: racks3
kube-prometheus-stack:
grafana:
adminUser: admin
adminPassword: admin123
stargate:
enabled: true
replicas: 1
heapMB: 256
cpuReqMillicores: 200
cpuLimMillicores: 1000
|
修改完之后，升级配置：
$ helm upgrade -f k8ssandra-values.yaml k8ssandra k8ssandra/k8ssandra
Release "k8ssandra" has been upgraded. Happy Helming!
NAME: k8ssandra
LAST DEPLOYED: Fri Oct 1 00:40:08 2021
NAMESPACE: default
STATUS: deployed
REVISION: 2
|
查看Kubernetes的相关资源：
3 查看监控
我们对外暴露Grafana服务来看看：
kubectl expose deployment k8ssandra-grafana --type=NodePort --name=grafana-out
|
找到对应的NodePort的端口30348，访问：http://外网IP:30348
账号：admin/admin123
界面如下，提供良好的监控界面：
4 总结
K8ssandra真是一个不错的开源项目，后续再介绍如何在开发中通过K8ssandra使用Cassandra吧。
代码请查看：https://github.com/LarryDpk/pkslow-samples
Reference:
Apache Cassandra Operations in Kubernetes
K8ssandra Get Started
Install K8ssandra on GKE
K8ssandra Helm Chart
用Helm部署Kubernetes应用，支持多环境部署与版本回滚
附：
使用到的镜像：
root@k8ssandra:~# docker images | sort | grep -v REPOSITORY | grep -v k8s.gcr.io
alpine 3.12.2 b14afc6dfb98 9 months ago 5.57MB
busybox 1.33.1 16ea53ea7c65 2 weeks ago 1.24MB
busybox latest 16ea53ea7c65 2 weeks ago 1.24MB
datastax/cass-config-builder 1.0.4 907e52ff2f78 4 months ago 354MB
gcr.io/k8s-minikube/storage-provisioner v5 6e38f40d628d 6 months ago 31.5MB
grafana/grafana 7.3.5 71716d95fc52 9 months ago 187MB
jettech/kube-webhook-certgen v1.5.0 344297e197b6 11 months ago 44.7MB
k8ssandra/cass-management-api 3.11.10-v0.1.27 e88cc3a8ce75 2 months ago 877MB
k8ssandra/cass-management-api 4.0.0-v0.1.28 59b9869dd160 7 weeks ago 726MB
k8ssandra/cass-operator v1.7.1 dd57363b0794 4 months ago 49.4MB
k8ssandra/k8ssandra-tools latest 43e4bd91cf9b 42 hours ago 49MB
k8ssandra/reaper-operator v0.3.3 b4cb9084a5e5 2 months ago 45.9MB
k8ssandra/system-logger 9c4c3692 4dda0db106cb 4 months ago 5.66MB
kiwigrid/k8s-sidecar 1.1.0 f4bb3bf1ddc7 11 months ago 82MB
quay.io/prometheus-operator/prometheus-config-reloader v0.44.0 73bc771f8028 10 months ago 13.4MB
quay.io/prometheus-operator/prometheus-operator v0.44.0 f5ab8d7a1a64 10 months ago 42.6MB
quay.io/prometheus/prometheus v2.22.1 7cc97b58fb0e 10 months ago 168MB
rancher/local-path-provisioner v0.0.20 933989e1174c 2 months ago 35MB
stargateio/stargate-3_11 v1.0.29 c276fffd9964 3 months ago 537MB
stargateio/stargate-4_0 v1.0.29 be4bae3748e1 3 months ago 546MB
thelastpickle/cassandra-reaper 2.3.1 8ad2c2784ead 2 months ago 418MB
|
|
