Skip to content
Snippets Groups Projects
Select Git revision
  • coding
  • master default protected
  • hyt
  • kafka_old
  • temp
5 results

kafka

  • Clone with SSH
  • Clone with HTTPS
  • user avatar
    关卫明 authored
    4e3c59fe
    History

    kafka配置参考

    https://xujiyou.work/%E5%A4%A7%E6%95%B0%E6%8D%AE/Kafka/Kubernetes%E4%B8%AD%E7%9A%84Kafka%E9%9B%86%E7%BE%A4%E5%A6%82%E4%BD%95%E5%9C%A8%E5%A4%96%E9%83%A8%E8%AE%BF%E9%97%AE.html

    删除pv

    kubectl patch pv kafka -p '{"metadata":{"finalizers":null}}' persistentvolume/kafka patched

    压测脚本

    进入kafka容器中

    /opt/kafka/bin/kafka-producer-perf-test.sh --topic demo1 --num-records 100000000 --record-size 100  --throughput 4000000 --producer-props bootstrap.servers=kafka1:9091,kafka2:9092,kafka3:9093
    /opt/kafka/bin/kafka-producer-perf-test.sh --topic demo1 --num-records 100000000 --record-size 20  --throughput 4000000 --producer-props bootstrap.servers=kafka1:9091,kafka2:9092,kafka3:9093

    ES吞吐量

    配置 32C64G SSD 每条日志 1KB
    kafka 写入峰值 472118.6 records/sec (450.25 MB/sec)
    ES 写入峰值 10W/秒