K8S 上部署大数据相关组件

文章目录

一、前言

  Artifact Hub 是一个专注于云原生应用的集中式搜索和发布平台。它旨在简化开发者在 CNCF(Cloud Native Computing Foundation)项目中寻找、安装和分享包与配置的过程。用户可以通过这个平台方便地发现、安装各类云原生工具的组件,如Argo模板、Helm 图表、Kubernetes 插件等。

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

二、Redis

$ helm repo add bitnami https://charts.bitnami.com/bitnami
"bitnami" has been added to your repositories

$ kubectl get sc
NAME                  PROVISIONER          RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
nfs                   nfs-provisioner-01   Retain          Immediate              false                  3d10h

# 执行安装命令并执行参数存储类的名称,得确保你集群中有这个存储类并且能够正常的提供PV动态供给
$ helm install redis-cluster oci://registry-1.docker.io/bitnamicharts/redis --set global.storageClass=nfs
Pulled: registry-1.docker.io/bitnamicharts/redis:19.6.1
Digest: sha256:0d077ee5947e26645e3bc05e3d6e6bd62e24e3082cf5df43d89664099336a78d
NAME: redis-cluster
LAST DEPLOYED: Mon Jul  8 03:42:57 2024
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
CHART NAME: redis
CHART VERSION: 19.6.1
APP VERSION: 7.2.5

** Please be patient while the chart is being deployed **

Redis® can be accessed on the following DNS names from within your cluster:

    redis-cluster-master.default.svc.cluster.local for read/write operations (port 6379)
    redis-cluster-replicas.default.svc.cluster.local for read-only operations (port 6379)



To get your password run:

    export REDIS_PASSWORD=$(kubectl get secret --namespace default redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)

To connect to your Redis® server:

1. Run a Redis® pod that you can use as a client:

   kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity

   Use the following command to attach to the pod:

   kubectl exec --tty -i redis-client \
   --namespace default -- bash

2. Connect using the Redis® CLI:
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-master
   REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-replicas

To connect to your database from outside the cluster execute the following commands:

    kubectl port-forward --namespace default svc/redis-cluster-master 6379:6379 &
    REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h 127.0.0.1 -p 6379

WARNING: There are "resources" sections in the chart not set. Using "resourcesPreset" is not recommended for production. For production installations, please set the following values according to your workload needs:
  - replica.resources
  - master.resources
+info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/

$ kubectl get pods
NAME                       READY   STATUS              RESTARTS   AGE
redis-cluster-master-0     1/1     Running             0          8m44s
redis-cluster-replicas-0   1/1     Running             0          8m44s
redis-cluster-replicas-1   1/1     Running             0          6m43s
redis-cluster-replicas-2   0/1     ContainerCreating   0          4m50s

# 将密码保存到 REDIS_PASSWORD 这个环境变量中
$ export REDIS_PASSWORD=$(kubectl get secret --namespace default redis-cluster -o jsonpath="{.data.redis-password}" | base64 -d)

# 再创建一个 Redis 客户端
$ kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity
pod/redis-client created

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
redis-client               1/1     Running   0          28s
redis-cluster-master-0     1/1     Running   0          10m
redis-cluster-replicas-0   1/1     Running   0          10m
redis-cluster-replicas-1   1/1     Running   0          8m50s
redis-cluster-replicas-2   1/1     Running   0          6m57s

$ kubectl run --namespace default redis-client --restart='Never'  --env REDIS_PASSWORD=$REDIS_PASSWORD  --image docker.io/bitnami/redis:7.2.5-debian-12-r2 --command -- sleep infinity
pod/redis-client created

$ kubectl get pods
NAME                       READY   STATUS    RESTARTS   AGE
redis-client               1/1     Running   0          28s
redis-cluster-master-0     1/1     Running   0          10m
redis-cluster-replicas-0   1/1     Running   0          10m
redis-cluster-replicas-1   1/1     Running   0          8m50s
redis-cluster-replicas-2   1/1     Running   0          6m57s

$ kubectl exec -it redis-client -- bash
I have no name!@redis-client:/$ REDISCLI_AUTH="$REDIS_PASSWORD" redis-cli -h redis-cluster-master
redis-cluster-master:6379> info replication
# Replication
role:master
connected_slaves:3
slave0:ip=redis-cluster-replicas-0.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
slave1:ip=redis-cluster-replicas-1.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
slave2:ip=redis-cluster-replicas-2.redis-cluster-headless.default.svc.cluster.local,port=6379,state=online,offset=924,lag=1
master_failover_state:no-failover
master_replid:239753ae516fad9e7fb6230d053861b80bc0bbb3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:924
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:924

$ kubectl get svc
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kubernetes               ClusterIP   10.68.0.1       <none>        443/TCP                               3d12h
redis-cluster-headless   ClusterIP   None            <none>        6379/TCP                              16m
redis-cluster-master     ClusterIP   10.68.240.7     <none>        6379/TCP                              16m
redis-cluster-replicas   ClusterIP   10.68.170.149   <none>        6379/TCP                              16m

  改为暴露端口从外部访问:

$ kubectl edit svc redis-cluster-master
# 原先内容
  ports:
  - name: tcp-redis
    port: 6379
    protocol: TCP
    targetPort: redis
  selector:
    app.kubernetes.io/component: master
    app.kubernetes.io/instance: redis-cluster
    app.kubernetes.io/name: redis
  sessionAffinity: None
  type: ClusterIP
# 修改为
  ports:
  - name: tcp-redis
    nodePort: 31379
    port: 6379
    protocol: TCP
    targetPort: redis
  selector:
    app.kubernetes.io/component: master
    app.kubernetes.io/instance: redis-cluster
    app.kubernetes.io/name: redis
  sessionAffinity: None
  type: NodePort
# 注意:nodePort 值不在指定范围会报错而无法保存(比如设置成 11379):
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
# services "redis-cluster-master" was not valid:
# * spec.ports[0].nodePort: Invalid value: 11379: provided port is not in the valid range. The range of valid ports is 30000-32767

# 重启服务(删除之后会自动重启)
$ kubectl delete svc redis-cluster-master -n default

$ kubectl get svc
NAME                     TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                               AGE
kubernetes               ClusterIP   10.68.0.1       <none>        443/TCP                               4d11h
redis-cluster-headless   ClusterIP   None            <none>        6379/TCP                              22h
redis-cluster-master     NodePort    10.68.240.7     <none>        6379:31379/TCP                        22h
redis-cluster-replicas   ClusterIP   10.68.170.149   <none>        6379/TCP                              22h

  补充:安装时参数自定义

# 安装的时候增加 namespace 的创建
$ helm install redis-cluster oci://registry-1.docker.io/bitnamicharts/redis -n redis --create-namespace --set global.storageClass=nfs

# redis 默认会安装1个master,3个node,可以通过以下参数来修改 --set replica.replicaCount=2 --set master.count=1
$ helm install --set replica.persistence.size=2Gi --set master.persistence.size=1Gi \
 --set global.storageClass=manual  --set  replica.replicaCount=2 --set master.count=1 linkage-redis bitnami/redis
 
# redis-cluster 默认创建6个 nodes(每个nodes包括一个master及一个replica),可以调整参数该边node数目,但调整后的nodes数不能<3
$ helm install --set cluster.nodes=3 --set replica.persistence.size=2Gi --set master.persistence.size=2Gi \
 --set global.storageClass=manual  linkage-redis bitnami/redis-cluster

参考这篇文章搭建出来的 pods 一直处于 pending 状态:k8s 如何访问redis集群

相关推荐

  1. 02-k8s学习笔记之相关组件

    2024-07-09 17:34:01       32 阅读
  2. k8s部署管理以及prometheus相关监控

    2024-07-09 17:34:01       48 阅读
  3. k8s部署 CNI 网络组件k8s集群搭建(二)

    2024-07-09 17:34:01       38 阅读
  4. K8S(三)—组件

    2024-07-09 17:34:01       42 阅读

最近更新

  1. Linux

    Linux

    2024-07-09 17:34:01      0 阅读
  2. 树上启发加点分治思想

    2024-07-09 17:34:01       0 阅读
  3. 配置文件格式 XML 快速上手

    2024-07-09 17:34:01       0 阅读
  4. Redis 数据结构—跳跃表(Skiplist)深度解析

    2024-07-09 17:34:01       0 阅读

热门阅读

  1. Qt提升控件失败的解决办法

    2024-07-09 17:34:01       6 阅读
  2. uniapp页面进来直接横屏

    2024-07-09 17:34:01       6 阅读
  3. Django权限系统如何使用?

    2024-07-09 17:34:01       5 阅读
  4. 开源 WAF 解析:选择最适合你的防护利器

    2024-07-09 17:34:01       8 阅读
  5. VPN是什么?

    2024-07-09 17:34:01       7 阅读
  6. Android C++系列:Linux进程(一)

    2024-07-09 17:34:01       10 阅读
  7. Oracle查询固定值查询

    2024-07-09 17:34:01       8 阅读
  8. android Gradle储蓄地址

    2024-07-09 17:34:01       10 阅读
  9. 基于BERT的大规模文本处理实战

    2024-07-09 17:34:01       7 阅读
  10. 【LeetCode 0242】【Map/排序】有效的异位词

    2024-07-09 17:34:01       8 阅读