1. 首頁 > 生活百科 > >

認識 Redis client-output-buffer-limit 參數與源碼分析

生活百科

概述Redis 的 client-output-buffer-limit 可以用來強制斷開無法足夠快從 redis 服務器端讀取數據的客戶端 。保護機制規則如下：

  1. [hard limit] 大小限制，當某一客戶端緩沖區超過設定值后，直接關閉連接 。
  2. [soft limit] 持續時間限制 ， 當某一客戶端緩沖區持續一段時間占用過大空間時關閉連接 。
該參數一般用在以下幾類客戶端中：
  • 普通 client ， 包括 monitor
  • 主從同步時的 slave client
  • Pub/Sub 模式中的 client
配置介紹與分析該參數的配置語法：
client-output-buffer-limit <class> <hard limit> <soft limit> <soft seconds>配置實例：
# 普通client buffer限制client-output-buffer-limit normal 0 0 0# slave client buffer限制client-output-buffer-limit slave 256mb 64mb 60# pubsub client buffer限制client-output-buffer-limit pubsub 32mb 8mb 60
  • client-output-buffer-limit normal 0 0 0
將 hard limit 和 soft limit 同時設置為 0，則表示關閉該限制 。
  • client-output-buffer-limit slave 256mb 64mb 60
該配置表示，對于 slave 客戶端來說，如果 output-buffer 占用內存達到 256M 或者超過 64M 的時間達到 60s ， 則關閉客戶端連接 。
  • client-output-buffer-limit pubsub 32mb 8mb 60
該配置表示，對于 Pub/Sub 客戶端來說，若 output-buffer 占用內存達到 32M 或者超過 8M 的時間達到 60s ， 則關閉客戶端連接 。
【認識 Redis client-output-buffer-limit 參數與源碼分析】概括說明：一般情況下，對于普通客戶端，client-output-buffer 是不設限制的，因為 server 只會在 client 請求數據的時候才會發送 ， 不會產生積壓 。而在 server 主動發送，client 來處理的場景下，這種一般都是異步處理的，會劃出一個緩沖區來“暫存”未處理的數據，若 server 發送數據比 client 處理數據快時，就會發生緩沖區積壓 。對于用作 Pub/Sub 和 slave 的客戶端，server 會主動把數據推送給他們，故需要設置 client-output-buffer 的限制 。
示例分析下面我們以主從同步時的 slave 客戶端，來具體分析下 。在 redis 在主從同步時 ， master 會為 slave 創建一個輸出緩沖區 。在 master 保存 rdb，將 rdb 文件傳輸給 slave，slave 加載 rdb 完成之前，master 會將接收到的所有寫命令 ， 寫入到內存中的這個輸出緩沖區去 。若 rdb 的保存，傳輸，加載耗時過長 ， 或者在此期間的寫命令過多，則可能會造成超過緩沖區限制，造成 master 和 slave 的連接斷開 。此時則需要適當調整下 client-output-buffer-limit slave配置 。
源碼淺析-主從同步時 output buffer 使用
基于 redis5.0 版本源碼
redis server 通過 addReply 將數據發送給客戶端，以下源碼見 https://github.com/redis/redis/blob/5.0/src/networking.c#L190-L211
/* Add the object 'obj' string representation to the client output buffer. */void addReply(client *c, robj *obj) {if (prepareClientToWrite(c) != C_OK) return;if (sdsEncodedObject(obj)) {if (_addReplyToBuffer(c,obj->ptr,sdslen(obj->ptr)) != C_OK)_addReplyStringToList(c,obj->ptr,sdslen(obj->ptr));} else if (obj->encoding == OBJ_ENCODING_INT) {char buf[32];size_t len = ll2string(buf,sizeof(buf),(long)obj->ptr);if (_addReplyToBuffer(c,buf,len) != C_OK)_addReplyStringToList(c,buf,len);} else {serverPanic("Wrong obj->encoding in addReply()");}}在函數的開頭，會通過prepareClientToWrite(c)判斷是否需要將數據寫入客戶端的 output buffer 中 。我們看下什么條件下數據會被寫入客戶端的 output buffer 中 ， 即返回 C_OK 。
/* This function is called every time we are going to transmit new data * to the client. The behavior is the following: * * If the client should receive new data (normal clients will) the function * returns C_OK, and make sure to install the write handler in our event * loop so that when the socket is writable new data gets written. * * If the client should not receive new data, because it is a fake client * (used to load AOF in memory), a master or because the setup of the write * handler failed, the function returns C_ERR. * * The function may return C_OK without actually installing the write * event handler in the following cases: * * 1) The event handler should already be installed since the output buffer *already contains something. * 2) The client is a slave but not yet online, so we want to just accumulate *writes in the buffer but not actually sending them yet. * * Typically gets called every time a reply is built, before adding more * data to the clients output buffers. If the function returns C_ERR no * data should be appended to the output buffers. */int prepareClientToWrite(client *c) {/* If it's the Lua client we always return ok without installing any* handler since there is no socket at all. */if (c->flags & (CLIENT_LUA|CLIENT_MODULE)) return C_OK;/* CLIENT REPLY OFF / SKIP handling: don't send replies. */if (c->flags & (CLIENT_REPLY_OFF|CLIENT_REPLY_SKIP)) return C_ERR;/* Masters don't receive replies, unless CLIENT_MASTER_FORCE_REPLY flag* is set. */if ((c->flags & CLIENT_MASTER) &&!(c->flags & CLIENT_MASTER_FORCE_REPLY)) return C_ERR;if (c->fd <= 0) return C_ERR; /* Fake client for AOF loading. *//* Schedule the client to write the output buffers to the socket, unless* it should already be setup to do so (it has already pending data). */if (!clientHasPendingReplies(c)) clientInstallWriteHandler(c);/* Authorize the caller to queue in the output buffer of this client. */return C_OK;}/* Return true if the specified client has pending reply buffers to write to * the socket. */int clientHasPendingReplies(client *c) {return c->bufpos || listLength(c->reply);}void clientInstallWriteHandler(client *c) {/* Schedule the client to write the output buffers to the socket only* if not already done and, for slaves, if the slave can actually receive* writes at this stage. */if (!(c->flags & CLIENT_PENDING_WRITE) &&(c->replstate == REPL_STATE_NONE ||(c->replstate == SLAVE_STATE_ONLINE && !c->repl_put_online_on_ack))){c->flags |= CLIENT_PENDING_WRITE;listAddNodeHead(server.clients_pending_write,c);}}

推薦閱讀