知乎架构变迁史

wn25

贡献于2015-01-05

字数:0 关键词: 软件架构

知乎架构变迁史 李申申 李申申是谁 RU 11M+ MAU 80M+ MPV 220M+ 2010.10 You jump off a cliff and you assemble an aeroplane on the way down. Reid Hoffman Linode 512M 你永远不知道明早醒来会面对什么问题 宕机@知乎 Redis M MySQL M MySQL S Redis S LVS LVS Nginx Nginx Web WebPicture Upload Picture Storage Mail Mailgun 经典问题MySQL 主从延迟 资源隔离 内网优化 硬盘升级 应用层灵活调用 Redis Shard 0/232 计算 hash node1 计算 hash Key 计算 hash node2 计算 hash node3 计算 hash Key 计算 hash node4 github.com/zhihu/redis-shard 工具是一个过程 用大小形状都合适的物体,以最有效的方式完成工作 Profiling Werkzeug Puppet Shipit 知乎邀请制 申请注册 日志系统 分布式收集 集中存储 实时 可订阅 简单 Scribe Kafka Flume 无法订阅 Scala Java Kids Kids is Data Stream Publish Publish Publish Subscribe SubscribePsubscribe App App App Agent Agent Agent Agent Kids Server Log Analyzer Log AnalyzerLog Analyzer Client Client Worker Thread Message Latest Message Latest Message Latest Client Client Worker Thread Storer Thread Master Thread App ( Publisher) MultipleStore BufferStore PriorityStore Store ⑤ Notify Worker Fetch Global Message Queue ⑦ Fetch ⑥ Notify Storer ④ Push Accept ② Accept ③ Publish ① Connect Connect Subscribe/Psubscribe⑧ Message App ( Subscriber) github.com/zhihu/kids 添加回答 更新通知 更新动态 更新搜索索引 判断是否 Spam 判断内容质量 更新回答计数 更新个人主页失效缓存 是否非法 内容审核? ? ? ? ? Event Driven Architecture Miller Master Beanstalkd Worker 01 Worker 02 Worker N Sink Miller Worker Web MySQL Sink noti-miller Master search-miller Master Beanstalkd noti-miller Worker search-miller Worker noti server indexer ① ② ③ ④ ④ ④ ⑤ ⑤ ⑥ ⑥ ⑦ ⑦ In 100+ Out 1500+ Miller Master Beanstalkd Worker 01 Worker 02 Worker N Sink Miller Worker Redis M MySQL M MySQL S Redis S LVS LVS Nginx Nginx Web Web 页面渲染优化 HomeUI FeedUI SidebarUI Item1UI Item2UI NavigatorUI QuestionUI AnswerUI Data HomeNode FeedNode SidebarNode Item1Node Item2Node NavigatorNode QuestionNode AnswerNode Data ZhihuNode class AnswerVoteBarV2(ZhihuNode): !    def init(self, answer_id):        self.answer_id = answer_id !    def template_v2(self):        return "v2_uimodule_answer_votebar.html" !    def prime(self):         models.AnswerDAO.prime_get_vote(self.login_id, self.answer_id) def obj(self): vote = models.AnswerDOA.get_vote(self.login_id, self.answer_id) return ObjectDict(vote=vote) Question 500ms -> 150ms Feed 1s -> 600ms Service-oriented architecture RPC 框架变迁史 TCP STP Protocol Buffers Wish TCP STP JSON Snow Zone Apache Avro IDL API Protocol Sync Client Async Client Connection Pool Async Server Protocol Versioning Communication & Transport Protocols Protocol Generator // Google Protocol Buffer message CalcResponse { enum Result { OK = 0; ZERO_DIVISION = 1; } required Result result = 1; optional int32 c = 2; } message CalcRequest { int32 a = 1; int32 b = 2; } service Calc { rpc Divide (CalcRequest) returns (CalcResponse); } Apache Avro - IDL // Apache Avro protocol Calc { error ZeroDivision { string message; } int divide(int a, int b) throws ZeroDivision; } // Wish’s generated files… xxx_pb2.py xxx_stub.py descriptor_pb2.py Apache Avro - API Protocol File // Apache Avro’s generated file, simply a snippet of JSON { "Calc": { "messages": { "divide": { "errors": ["ZeroDivision"], "reqest": [{"name": "a", "type": "int"}, {"name": "b", "type": "int"}], "response": "int"}}, "namespace": None, "protocol": "Calc", "types": [ {"fields": [{"name": "message", "type": "string"}], "name": "ZeroDivision", "type": "error"} ] } } import snow class Calc(snow.APIBase): def divide(self, a, b): if b == 0: return {"code": 1, "msg": "Zero division."} return {"code": 0, "c": a / b} ! snow.Server(Calc(), port=9000).run() Server Example import zone from proto import protos ! class Calc(zone.ZoneAPI): ! def divide(self, a, b): if b == 0: raise zone.ZoneCustomException(name="ZeroDivision") return a / b ! zone.Server([Calc(protos["Calc"])], port=9000).run() TCP Transporter (Binary/STP) Serializer (Avro / JSON) Header Body Consul Tracing 知乎业务服务结构图 话题 Feed ⾸⻚ Feed 发现 搜索 推荐 问答 专栏 通知 私信 已读 ⽤户 评论 图⽚ 推送 分享 收藏 圆桌 话题 邮件 短址 聚 合 层 内 容 层 基 础 层 数据服务 逻辑服务 通道服务 LVS01 LVS02 Nginx01 Nginx02 Web01 Web02 WebN keepAlived Beantalk Haproxy01 Haproxy02keepAlived Feed Service Member Service ···Miller Sink Kids MySQL Redis Cassandra 前端展⽰层 后端业务逻辑层 存储 &通信层 知乎专栏的全新实践 传统网站的开发模式 前后端代码交叉 工程师互相等待 前端代码部署成本高 Web App RESTful API Server RPC Server Workers MySQL Redis 前后端分离,逻辑更清晰 AngularJS:代码复用、快速开发 并行开发 独立部署 产品设计师也能修改代码了 <(▰˘◡˘▰)> 工具要不断跟上 Dash Heroin Radius CFB Crony Oops Boxen 想各种办法发挥自我创新 其实就是找各种办法刺激工程师们 Hackathon Tink Day 谢谢

下载文档,方便阅读与编辑

文档的实际排版效果,会与网站的显示效果略有不同!!

需要 6 金币 [ 分享文档获得金币 ] 1 人已下载

下载文档

相关文档