【2022年03月】爱提网最新架构介绍

幽星人
幽星人 2022-03-20 18:43:47
来源:爱提网
itgoodboy.com网站最新架构:

(一)简介:
1.整个网站基于go语言开发(gin框架)和go template,前端使用layui框架。【2021年6月】
2.使用了mysql,es,redis,oss等服务,nginx做负载均衡。
3.使用docker容器简化服务部署。【2021年4月】
4.使用阿里canal中间件作为mysql与es数据同步的工具(注意:adapter1.1.5release版本中的es.jar包有bug,需要用alpha-2版本里的jar包)。【2022年3月20日】
5.前台文章数据均从es/redis读取,大大提高了访问速度。

(二)功能介绍:
web:
go(gin框架)

supervisor配置:
/etc/supervisord.d/gin-******mand=/root/go/src/gin-******=true
autorestart=true
stderr_logfile=/tmp/go/gin-stderr.log
stdout_logfile=/tmp/go/gin-stdout.log
user = rootweb服务:
go build -o ./bin/main main.go
计划任务:
go build -o ./bin/news-check ./app/console/news_check.go
mysql: 阿里云RDS

redis: 使用docker镜像

es: /home/elk/elastic/elasticsearch-7.2.0/config/elasticsearch.yml
node.name: node-1
network.host: 0.0.0.0
http.port: 9200
xpack.security.enabled: true
xpack.security.authc.accept_default_password: false
discovery.type: single-node
filebeat: /home/elk/elastic/filebeat-7.2.0/filebeat.yml
#================================ Outputs =====================================
# Configure what output to use when sending the data collected by the beat.
#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["0.0.0.0"]
# Optional protocol and basic auth credentials.
#protocol: "https"
username: "xxx"
password: "xxx"
kibana: /home/elk/elastic/kibana-7.2.0/config/kibana.yml
server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ******: "xxx"
elasticsearch.password: "xxx"
xpack.security.encryptionKey: "something_at_least_32_characters"
canal:mysql数据同步到es的工具
canal服务端配置:
/web/canal/1.1.5/deployer/conf/example/instance.properties
# position info
canal.instance.master.address=xxx:3306
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=
canal.instance.master.gtid=

# username/password
canal.instance.dbUsername=xxx
canal.instance.dbPassword=xxx
canal.instance.connectionCharset = UTF-8

# table regex
canal.instance.filter.regex=xxx.xxx
canal客户端配置:
/web/canal/1.1.5/adapter/conf/application.yml
srcDataSources:
defaultDS:
url: jdbc:mysql://xxx:3306/xxx?useUnicode=true
username: canal
password: xxx
canalAdapters:
- instance: example # canal instance Name or mq topic name
groups:
- groupId: g1
outerAdapters:
- name: logger
- name: es7
hosts: xxx:9200 # 127.0.0.1:9200 for rest mode
properties:
mode: rest # transport # or rest
security.auth: username:password # only used for rest mode
cluster.name: elasticsearch # my-application # elasticsearch
/web/canal/1.1.5/adapter/conf/es7/article.yml
dataSourceKey: defaultDS # 源数据源的key, 对应上面配置的srcDataSources中的值
destination: example # canal的instance或者MQ的topic
groupId: g1 # 对应MQ模式下的groupId, 只会同步对应groupId的数据
esMapping:
_index: article_xxx # es 的索引名称
_id: _id # es 的_id, 如果不配置该项必须配置下面的pk项_id则会由es自动分配
sql: "SELECT
a.id AS _id,
a.newsid,
a.uuid,
a.userid,
a.parentid,
a.catid,
a.title,
a.author,
a.keyword,
a.thumbid,
a.status,
a.inputtime,
a.updatetime,
a.ctime
FROM
article AS a"
#etlCondition: "where a.updatetime>={}" #etl的条件参数
commitBatch: 1000 # 提交批大小
后续会持续更新。
标签: 网站架构