Redis Pipelining 底層原理分析及實(shí)踐
Redis是一種基于客戶(hù)端-服務(wù)端模型以及請(qǐng)求/響應(yīng)的TCP服務(wù)。在遇到批處理命令執(zhí)行時(shí)，Redis提供了Pipelining(管道)來(lái)提升批處理性能。本文結(jié)合實(shí)踐分析了Spring Boot框架下Redis的Lettuce客戶(hù)端和Redisson客戶(hù)端對(duì)Pipeline特性的支持原理，并針對(duì)實(shí)踐過(guò)程中遇到的問(wèn)題進(jìn)行了分析，可以幫助開(kāi)發(fā)者了解不同客戶(hù)端對(duì)Pipeline支持原理及避免實(shí)際使用中出現(xiàn)問(wèn)題。
一、前言
Redis 已經(jīng)提供了像 mget 、mset 這種批量的命令，但是某些操作根本就不支持或沒(méi)有批量的操作，從而與 Redis 高性能背道而馳。為此, Redis基于管道機(jī)制，提供Redis Pipeline新特性。Redis Pipeline是一種通過(guò)一次性發(fā)送多條命令并在執(zhí)行完后一次性將結(jié)果返回，從而減少客戶(hù)端與redis的通信次數(shù)來(lái)實(shí)現(xiàn)降低往返延時(shí)時(shí)間提升操作性能的技術(shù)。目前，Redis Pipeline是被很多個(gè)版本的Redis 客戶(hù)端所支持的。
二、Pipeline 底層原理分析
2.1 Redis單個(gè)命令執(zhí)行基本步驟
Redis是一種基于客戶(hù)端-服務(wù)端模型以及請(qǐng)求/響應(yīng)的TCP服務(wù)。一次Redis客戶(hù)端發(fā)起的請(qǐng)求，經(jīng)過(guò)服務(wù)端的響應(yīng)后，大致會(huì)經(jīng)歷如下的步驟：
- 客戶(hù)端發(fā)起一個(gè)（查詢(xún)/插入）請(qǐng)求，并監(jiān)聽(tīng)socket返回，通常情況都是阻塞模式等待Redis服務(wù)器的響應(yīng)。
- 服務(wù)端處理命令，并且返回處理結(jié)果給客戶(hù)端。
- 客戶(hù)端接收到服務(wù)的返回結(jié)果，程序從阻塞代碼處返回。
2.2 RTT 時(shí)間
Redis客戶(hù)端和服務(wù)端之間通過(guò)網(wǎng)絡(luò)連接進(jìn)行數(shù)據(jù)傳輸，數(shù)據(jù)包從客戶(hù)端到達(dá)服務(wù)器，并從服務(wù)器返回?cái)?shù)據(jù)回復(fù)客戶(hù)端的時(shí)間被稱(chēng)之為RTT(Round Trip Time - 往返時(shí)間)。我們可以很容易就意識(shí)到，Redis在連續(xù)請(qǐng)求服務(wù)端時(shí)，如果RTT時(shí)間為250ms, 即使Redis每秒能處理100k請(qǐng)求，但也會(huì)因?yàn)榫W(wǎng)絡(luò)傳輸花費(fèi)大量時(shí)間，導(dǎo)致每秒最多也只能處理4個(gè)請(qǐng)求，導(dǎo)致整體性能的下降。
2.3 Redis Pipeline
為了提升效率，這時(shí)候Pipeline出現(xiàn)了。Pipelining不僅僅能夠降低RRT，實(shí)際上它極大的提升了單次執(zhí)行的操作數(shù)。這是因?yàn)槿绻皇褂肞ipelining，那么每次執(zhí)行單個(gè)命令,從訪(fǎng)問(wèn)數(shù)據(jù)的結(jié)構(gòu)和服務(wù)端產(chǎn)生應(yīng)答的角度，它的成本是很低的。但是從執(zhí)行網(wǎng)絡(luò)IO的角度，它的成本其實(shí)是很高的。其中涉及到read()和write()的系統(tǒng)調(diào)用，這意味著需要從用戶(hù)態(tài)切換到內(nèi)核態(tài),而這個(gè)上下文的切換成本是巨大的。
當(dāng)使用Pipeline時(shí)，它允許多個(gè)命令的讀通過(guò)一次read()操作，多個(gè)命令的應(yīng)答使用一次write()操作，它允許客戶(hù)端可以一次發(fā)送多條命令，而不等待上一條命令執(zhí)行的結(jié)果。不僅減少了RTT，同時(shí)也減少了IO調(diào)用次數(shù)（IO調(diào)用涉及到用戶(hù)態(tài)到內(nèi)核態(tài)之間的切換），最終提升程序的執(zhí)行效率與性能。如下圖：
要支持Pipeline，其實(shí)既要服務(wù)端的支持，也要客戶(hù)端支持。對(duì)于服務(wù)端來(lái)說(shuō)，所需要的是能夠處理一個(gè)客戶(hù)端通過(guò)同一個(gè)TCP連接發(fā)來(lái)的多個(gè)命令，可以理解為，這里將多個(gè)命令切分，和處理單個(gè)命令一樣，Redis就是這樣處理的。而客戶(hù)端，則是要將多個(gè)命令緩存起來(lái)，緩沖區(qū)滿(mǎn)了就發(fā)送，然后再寫(xiě)緩沖，最后才處理Redis的應(yīng)答。
三、Pipeline 基本使用及性能比較
下面我們以給10w個(gè)set結(jié)構(gòu)分別插入一個(gè)整數(shù)值為例，分別使用jedis單個(gè)命令插入、jedis使用Pipeline模式進(jìn)行插入和redisson使用Pipeline模式進(jìn)行插入以及測(cè)試其耗時(shí)。
@Slf4j
public class RedisPipelineTestDemo {
public static void main(String[] args) {
//連接redis
Jedis jedis = new Jedis("10.101.17.180", 6379);
//jedis逐一給每個(gè)set新增一個(gè)value
String zSetKey = "Pipeline-test-set";
int size = 100000;
long begin = System.currentTimeMillis();
for (int i = 0; i < size; i++) {
jedis.sadd(zSetKey + i, "aaa");
}
log.info("Jedis逐一給每個(gè)set新增一個(gè)value耗時(shí)：{}ms", (System.currentTimeMillis() - begin));
//Jedis使用Pipeline模式 Pipeline Pipeline = jedis.Pipelined();
begin = System.currentTimeMillis();
for (int i = 0; i < size; i++) { Pipeline.sadd(zSetKey + i, "bbb");
} Pipeline.sync();
log.info("Jedis Pipeline模式耗時(shí)：{}ms", (System.currentTimeMillis() - begin));
//Redisson使用Pipeline模式
Config config = new Config();
config.useSingleServer().setAddress("redis://10.101.17.180:6379");
RedissonClient redisson = Redisson.create(config);
RBatch redisBatch = redisson.createBatch();
begin = System.currentTimeMillis();
for (int i = 0; i < size; i++) {
redisBatch.getSet(zSetKey + i).addAsync("ccc");
}
redisBatch.execute();
log.info("Redisson Pipeline模式耗時(shí)：{}ms", (System.currentTimeMillis() - begin));
//關(guān)閉 Pipeline.close();
jedis.close();
redisson.shutdown();
}
}測(cè)試結(jié)果如下:
Jedis逐一給每個(gè)set新增一個(gè)value耗時(shí)：162655ms
Jedis Pipeline模式耗時(shí)：504ms
Redisson Pipeline模式耗時(shí)：1399ms
我們發(fā)現(xiàn)使用Pipeline模式對(duì)應(yīng)的性能會(huì)明顯好于單個(gè)命令執(zhí)行的情況。
四、項(xiàng)目中實(shí)際應(yīng)用
在實(shí)際使用過(guò)程中有這樣一個(gè)場(chǎng)景，很多應(yīng)用在節(jié)假日的時(shí)候需要更新應(yīng)用圖標(biāo)樣式，在運(yùn)營(yíng)進(jìn)行后臺(tái)配置的時(shí)候, 可以根據(jù)圈選的用戶(hù)標(biāo)簽預(yù)先計(jì)算出單個(gè)用戶(hù)需要下發(fā)的圖標(biāo)樣式并存儲(chǔ)在Redis里面，從而提升性能，這里就涉及Redis的批量操作問(wèn)題，業(yè)務(wù)流程如下：
為了提升Redis操作性能，我們決定使用Redis Pipelining機(jī)制進(jìn)行批量執(zhí)行。
4.1 Redis 客戶(hù)端對(duì)比
針對(duì)Java技術(shù)棧而言，目前Redis使用較多的客戶(hù)端為Jedis、Lettuce和Redisson。
目前項(xiàng)目主要是基于SpringBoot開(kāi)發(fā)，針對(duì)Redis，其默認(rèn)的客戶(hù)端為L(zhǎng)ettuce，所以我們基于Lettuce客戶(hù)端進(jìn)行分析。
4.2 Spring環(huán)境下Lettuce客戶(hù)端對(duì)Pipeline的實(shí)現(xiàn)
在Spring環(huán)境下，使用Redis的Pipeline也是很簡(jiǎn)單的。spring-data-redis提供了
StringRedisTemplate簡(jiǎn)化了對(duì)Redis的操作, 只需要調(diào)用StringRedisTemplate的executePipelined方法就可以了，但是在參數(shù)中提供了兩種回調(diào)方式：SessionCallback和RedisCallback。
兩種使用方式如下(這里以操作set結(jié)構(gòu)為例)：
RedisCallback的使用方式：
public void testRedisCallback() {
List<Integer> ids= Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9);
Integer contentId = 1;
redisTemplate.executePipelined(new InsertPipelineExecutionA(ids, contentId));
}
@AllArgsConstructor
private static class InsertPipelineExecutionA implements RedisCallback<Void> {
private final List<Integer> ids;
private final Integer contentId;
@Override
public Void doInRedis(RedisConnection connection) DataAccessException {
RedisSetCommands redisSetCommands = connection.setCommands();
ids.forEach(id-> {
String redisKey = "aaa:" + id;
String value = String.valueOf(contentId);
redisSetCommands.sAdd(redisKey.getBytes(), value.getBytes());
});
return null;
}
}SessionCallback的使用方式：
public void testSessionCallback() {
List<Integer> ids= Arrays.asList(1, 2, 3, 4, 5, 6, 7, 8, 9);
Integer contentId = 1;
redisTemplate.executePipelined(new InsertPipelineExecutionB(ids, contentId));
}
@AllArgsConstructor
private static class InsertPipelineExecutionB implements SessionCallback<Void> {
private final List<Integer> ids;
private final Integer contentId;
@Override
public <K, V> Void execute(RedisOperations<K, V> operations) throws DataAccessException {
SetOperations<String, String> setOperations = (SetOperations<String, String>) operations.opsForSet();
ids.forEach(id-> {
String redisKey = "aaa:" + id;
String value = String.valueOf(contentId);
setOperations.add(redisKey, value);
});
return null;
}
}4.3 RedisCallBack和SessionCallback之間的比較
1、RedisCallBack和SessionCallback都可以實(shí)現(xiàn)回調(diào)，通過(guò)它們可以在同一條連接中一次執(zhí)行多個(gè)redis命令。
2、RedisCallback使用的是原生
RedisConnection，用起來(lái)比較麻煩，比如上面執(zhí)行set的add操作，key和value需要進(jìn)行轉(zhuǎn)換，可讀性差，但原生api提供的功能比較齊全。
3、SessionCalback提供了良好的封裝，可以?xún)?yōu)先選擇使用這種回調(diào)方式。
最終的代碼實(shí)現(xiàn)如下：
public void executeB(List<Integer> userIds, Integer iconId) {
redisTemplate.executePipelined(new InsertPipelineExecution(userIds, iconId));
}
@AllArgsConstructor
private static class InsertPipelineExecution implements SessionCallback<Void> {
private final List<Integer> userIds;
private final Integer iconId;
@Override
public <K, V> Void execute(RedisOperations<K, V> operations) throws DataAccessException {
SetOperations<String, String> setOperations = (SetOperations<String, String>) operations.opsForSet();
userIds.forEach(userId -> {
String redisKey = "aaa:" + userId;
String value = String.valueOf(iconId);
setOperations.add(redisKey, value);
});
return null;
}
}4.4 源碼分析
那么為什么使用Pipeline方式會(huì)對(duì)性能有較大提升呢，我們現(xiàn)在從源碼入手著重分析一下：
4.4.1 Pipeline方式下獲取連接相關(guān)原理分析：
@Override
public List<Object> executePipelined(SessionCallback<?> session, @Nullable RedisSerializer<?> resultSerializer) {
Assert.isTrue(initialized, "template not initialized; call afterPropertiesSet() before using it");
Assert.notNull(session, "Callback object must not be null");
//1. 獲取對(duì)應(yīng)的Redis連接工廠
RedisConnectionFactory factory = getRequiredConnectionFactory();
//2. 綁定連接過(guò)程
RedisConnectionUtils.bindConnection(factory, enableTransactionSupport);
try {
//3. 執(zhí)行命令流程, 這里請(qǐng)求參數(shù)為RedisCallback, 里面有對(duì)應(yīng)的回調(diào)操作
return execute((RedisCallback<List<Object>>) connection -> {
//具體的回調(diào)邏輯
connection.openPipeline();
boolean PipelinedClosed = false;
try {
//執(zhí)行命令
Object result = executeSession(session);
if (result != null) {
throw new InvalidDataAccessApiUsageException(
"Callback cannot return a non-null value as it gets overwritten by the Pipeline");
}
List<Object> closePipeline = connection.closePipeline(); PipelinedClosed = true;
return deserializeMixedResults(closePipeline, resultSerializer, hashKeySerializer, hashValueSerializer);
} finally {
if (!PipelinedClosed) {
connection.closePipeline();
}
}
});
} finally {
RedisConnectionUtils.unbindConnection(factory);
}
}① 獲取對(duì)應(yīng)的Redis連接工廠，這里要使用Pipeline特性需要使用
LettuceConnectionFactory方式，這里獲取的連接工廠就是LettuceConnectionFactory。
② 綁定連接過(guò)程，具體指的是將當(dāng)前連接綁定到當(dāng)前線(xiàn)程上面， 核心方法為：doGetConnection。
public static RedisConnection doGetConnection(RedisConnectionFactory factory, boolean allowCreate, boolean bind,
boolean enableTransactionSupport) {
Assert.notNull(factory, "No RedisConnectionFactory specified");
//核心類(lèi)，有緩存作用，下次可以從這里獲取已經(jīng)存在的連接
RedisConnectionHolder connHolder = (RedisConnectionHolder) TransactionSynchronizationManager.getResource(factory);
//如果connHolder不為null, 則獲取已經(jīng)存在的連接, 提升性能
if (connHolder != null) {
if (enableTransactionSupport) {
potentiallyRegisterTransactionSynchronisation(connHolder, factory);
}
return connHolder.getConnection();
}
......
//第一次獲取連接，需要從Redis連接工廠獲取連接
RedisConnection conn = factory.getConnection();
//bind = true 執(zhí)行綁定
if (bind) {
RedisConnection connectionToBind = conn;
......
connHolder = new RedisConnectionHolder(connectionToBind);
//綁定核心代碼: 將獲取的連接和當(dāng)前線(xiàn)程綁定起來(lái)
TransactionSynchronizationManager.bindResource(factory, connHolder);
......
return connHolder.getConnection();
}
return conn;
}里面有個(gè)核心類(lèi)RedisConnectionHolder，我們看一下
RedisConnectionHolder connHolder =
(RedisConnectionHolder)
TransactionSynchronizationManager.getResource(factory);
@Nullable
public static Object getResource(Object key) {
Object actualKey = TransactionSynchronizationUtils.unwrapResourceIfNecessary(key);
Object value = doGetResource(actualKey);
if (value != null && logger.isTraceEnabled()) {
logger.trace("Retrieved value [" + value + "] for key [" + actualKey + "] bound to thread [" +
Thread.currentThread().getName() + "]");
}
return value;
}里面有一個(gè)核心方法doGetResource
(actualKey)，大家很容易猜測(cè)這里涉及到一個(gè)map結(jié)構(gòu)，如果我們看源碼，也確實(shí)是這樣一個(gè)結(jié)構(gòu)。
@Nullable
private static Object doGetResource(Object actualKey) {
Map<Object, Object> map = resources.get();
if (map == null) {
return null;
}
Object value = map.get(actualKey);
// Transparently remove ResourceHolder that was marked as void...
if (value instanceof ResourceHolder && ((ResourceHolder) value).isVoid()) {
map.remove(actualKey);
// Remove entire ThreadLocal if empty...
if (map.isEmpty()) {
resources.remove();
}
value = null;
}
return value;
}resources是一個(gè)ThreadLocal類(lèi)型，這里會(huì)涉及到根據(jù)RedisConnectionFactory獲取到連接connection的邏輯,如果下一次是同一個(gè)actualKey，那么就直接使用已經(jīng)存在的連接，而不需要新建一個(gè)連接。第一次這里map為null，就直接返回了，然后回到doGetConnection方法，由于這里bind為true，我們會(huì)執(zhí)行TransactionSynchronizationManager.bindResource(factory, connHolder);，也就是將連接和當(dāng)前線(xiàn)程綁定了起來(lái)。
public static void bindResource(Object key, Object value) throws IllegalStateException {
Object actualKey = TransactionSynchronizationUtils.unwrapResourceIfNecessary(key);
Assert.notNull(value, "Value must not be null");
Map<Object, Object> map = resources.get();
// set ThreadLocal Map if none found
if (map == null) {
map = new HashMap<>();
resources.set(map);
}
Object oldValue = map.put(actualKey, value);
......
}③ 我們回到executePipelined，在獲取到連接工廠，將連接和當(dāng)前線(xiàn)程綁定起來(lái)以后，就開(kāi)始需要正式去執(zhí)行命令了, 這里會(huì)調(diào)用execute方法
@Override
@Nullable
public <T> T execute(RedisCallback<T> action) {
return execute(action, isExposeConnection());
}這里我們注意到execute方法的入?yún)镽edisCallback<T>action，RedisCallback對(duì)應(yīng)的doInRedis操作如下,這里在后面的調(diào)用過(guò)程中會(huì)涉及到回調(diào)。
connection.openPipeline();
boolean PipelinedClosed = false;
try {
Object result = executeSession(session);
if (result != null) {
throw new InvalidDataAccessApiUsageException(
"Callback cannot return a non-null value as it gets overwritten by the Pipeline");
}
List<Object> closePipeline = connection.closePipeline(); PipelinedClosed = true;
return deserializeMixedResults(closePipeline, resultSerializer, hashKeySerializer, hashValueSerializer);
} finally {
if (!PipelinedClosed) {
connection.closePipeline();
}
}我們?cè)賮?lái)看execute(action,
isExposeConnection())方法，這里最終會(huì)調(diào)用
<T>execute(RedisCallback<T>action, boolean exposeConnection, boolean Pipeline)方法。
@Nullable
public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean Pipeline) {
Assert.isTrue(initialized, "template not initialized; call afterPropertiesSet() before using it");
Assert.notNull(action, "Callback object must not be null");
//獲取對(duì)應(yīng)的連接工廠
RedisConnectionFactory factory = getRequiredConnectionFactory();
RedisConnection conn = null;
try {
if (enableTransactionSupport) {
// only bind resources in case of potential transaction synchronization
conn = RedisConnectionUtils.bindConnection(factory, enableTransactionSupport);
} else {
//獲取對(duì)應(yīng)的連接(enableTransactionSupport=false)
conn = RedisConnectionUtils.getConnection(factory);
}
boolean existingConnection = TransactionSynchronizationManager.hasResource(factory);
RedisConnection connToUse = preProcessConnection(conn, existingConnection);
boolean PipelineStatus = connToUse.isPipelined();
if (Pipeline && !PipelineStatus) {
connToUse.openPipeline();
}
RedisConnection connToExpose = (exposeConnection ? connToUse : createRedisConnectionProxy(connToUse));
//核心方法，這里就開(kāi)始執(zhí)行回調(diào)操作
T result = action.doInRedis(connToExpose);
// close Pipeline
if (Pipeline && !PipelineStatus) {
connToUse.closePipeline();
}
// TODO: any other connection processing?
return postProcessResult(result, connToUse, existingConnection);
} finally {
RedisConnectionUtils.releaseConnection(conn, factory, enableTransactionSupport);
}
}我們看到這里最開(kāi)始也是獲取對(duì)應(yīng)的連接工廠，然后獲取對(duì)應(yīng)的連接
(enableTransactionSupport=false)，具體調(diào)用是
RedisConnectionUtils.getConnection(factory)方法，最終會(huì)調(diào)用
RedisConnection doGetConnection(RedisConnectionFactory factory, boolean allowCreate, boolean bind, boolean enableTransactionSupport)，此時(shí)bind為false
public static RedisConnection doGetConnection(RedisConnectionFactory factory, boolean allowCreate, boolean bind,
boolean enableTransactionSupport) {
Assert.notNull(factory, "No RedisConnectionFactory specified");
//直接獲取與當(dāng)前線(xiàn)程綁定的Redis連接
RedisConnectionHolder connHolder = (RedisConnectionHolder) TransactionSynchronizationManager.getResource(factory);
if (connHolder != null) {
if (enableTransactionSupport) {
potentiallyRegisterTransactionSynchronisation(connHolder, factory);
}
return connHolder.getConnection();
}
......
return conn;
}前面我們分析過(guò)一次，這里調(diào)用
RedisConnectionHolder connHolder =
(RedisConnectionHolder) TransactionSynchronizationManager.getResource(factory);會(huì)獲取到之前和當(dāng)前線(xiàn)程綁定的Redis，而不會(huì)新創(chuàng)建一個(gè)連接。
然后會(huì)去執(zhí)行T result = action.
doInRedis(connToExpose)，這里的action為RedisCallback，執(zhí)行doInRedis為：
//開(kāi)啟Pipeline功能
connection.openPipeline();
boolean PipelinedClosed = false;
try {
//執(zhí)行Redis命令
Object result = executeSession(session);
if (result != null) {
throw new InvalidDataAccessApiUsageException(
"Callback cannot return a non-null value as it gets overwritten by the Pipeline");
}
List<Object> closePipeline = connection.closePipeline(); PipelinedClosed = true;
return deserializeMixedResults(closePipeline, resultSerializer, hashKeySerializer, hashValueSerializer);
} finally {
if (!PipelinedClosed) {
connection.closePipeline();
}
}這里最開(kāi)始會(huì)開(kāi)啟Pipeline功能，然后執(zhí)行
Object result = executeSession(session);
private Object executeSession(SessionCallback<?> session) {
return session.execute(this);
}這里會(huì)調(diào)用我們自定義的execute方法
@AllArgsConstructor
private static class InsertPipelineExecution implements SessionCallback<Void> {
private final List<Integer> userIds;
private final Integer iconId;
@Override
public <K, V> Void execute(RedisOperations<K, V> operations) throws DataAccessException {
SetOperations<String, String> setOperations = (SetOperations<String, String>) operations.opsForSet();
userIds.forEach(userId -> {
String redisKey = "aaa:" + userId;
String value = String.valueOf(iconId);
setOperations.add(redisKey, value);
});
return null;
}
}進(jìn)入到foreach循環(huán)，執(zhí)行DefaultSetOperations的add方法。
@Override
public Long add(K key, V... values) {
byte[] rawKey = rawKey(key);
byte[][] rawValues = rawValues((Object[]) values);
//這里的connection.sAdd是后續(xù)回調(diào)要執(zhí)行的方法
return execute(connection -> connection.sAdd(rawKey, rawValues), true);
}這里會(huì)繼續(xù)執(zhí)行redisTemplate的execute方法，里面最終會(huì)調(diào)用我們之前分析過(guò)的<T>T execute(RedisCallback<T>action, boolean exposeConnection, boolean Pipeline)方法。
@Nullable
public <T> T execute(RedisCallback<T> action, boolean exposeConnection, boolean Pipeline) {
Assert.isTrue(initialized, "template not initialized; call afterPropertiesSet() before using it");
Assert.notNull(action, "Callback object must not be null");
RedisConnectionFactory factory = getRequiredConnectionFactory();
RedisConnection conn = null;
try {
......
//再次執(zhí)行回調(diào)方法，這里執(zhí)行的Redis基本數(shù)據(jù)結(jié)構(gòu)對(duì)應(yīng)的操作命令
T result = action.doInRedis(connToExpose);
......
// TODO: any other connection processing?
return postProcessResult(result, connToUse, existingConnection);
} finally {
RedisConnectionUtils.releaseConnection(conn, factory, enableTransactionSupport);
}
}這里會(huì)繼續(xù)執(zhí)行T result =
action.doInRedis(connToExpose);，這里其實(shí)執(zhí)行的doInRedis方法為：
connection -> connection.sAdd(rawKey, rawValues)4.4.2 Pipeline方式下執(zhí)行命令的流程分析:
① 接著上面的流程分析，這里的sAdd方法實(shí)際調(diào)用的是DefaultStringRedisConnection的sAdd方法
@Override
public Long sAdd(byte[] key, byte[]... values) {
return convertAndReturn(delegate.sAdd(key, values), identityConverter);
}② 這里會(huì)進(jìn)一步調(diào)用
DefaultedRedisConnection的sAdd方法
@Override
@Deprecated
default Long sAdd(byte[] key, byte[]... values) {
return setCommands().sAdd(key, values);
}③ 接著調(diào)用LettuceSetCommands的sAdd方法
@Override
public Long sAdd(byte[] key, byte[]... values) {
Assert.notNull(key, "Key must not be null!");
Assert.notNull(values, "Values must not be null!");
Assert.noNullElements(values, "Values must not contain null elements!");
try {
// 如果開(kāi)啟了 Pipelined 模式，獲取的是 異步連接，進(jìn)行異步操作
if (isPipelined()) { Pipeline(connection.newLettuceResult(getAsyncConnection().sadd(key, values)));
return null;
}
if (isQueueing()) {
transaction(connection.newLettuceResult(getAsyncConnection().sadd(key, values)));
return null;
}
//常規(guī)模式下，使用的是同步操作
return getConnection().sadd(key, values);
} catch (Exception ex) {
throw convertLettuceAccessException(ex);
}
}這里我們開(kāi)啟了Pipeline, 實(shí)際會(huì)調(diào)用
Pipeline(connection.newLettuceResult(getAsyncConnection().sadd(key, values))); 也就是獲取異步連接getAsyncConnection，然后進(jìn)行異步操作sadd，而常規(guī)模式下，使用的是同步操作，所以在Pipeline模式下，執(zhí)行效率更高。
從上面的獲取連接和具體命令執(zhí)行相關(guān)源碼分析可以得出使用Lettuce客戶(hù)端Pipeline模式高效的根本原因：
- 普通模式下，每執(zhí)行一個(gè)命令都需要先打開(kāi)一個(gè)連接，命令執(zhí)行完畢以后又需要關(guān)閉這個(gè)連接，執(zhí)行下一個(gè)命令時(shí)，又需要經(jīng)過(guò)連接打開(kāi)和關(guān)閉的流程；而Pipeline的所有命令的執(zhí)行只需要經(jīng)過(guò)一次連接打開(kāi)和關(guān)閉。
- 普通模式下命令的執(zhí)行是同步阻塞模式，而Pipeline模式下命令的執(zhí)行是異步非阻塞模式。
五、項(xiàng)目中遇到的坑
前面介紹了涉及到批量操作，可以使用Redis Pipelining機(jī)制，那是不是任何批量操作相關(guān)的場(chǎng)景都可以使用呢，比如list類(lèi)型數(shù)據(jù)的批量移除操作，我們的代碼最開(kāi)始是這么寫(xiě)的：
public void deleteSet(String updateKey, Set<Integer> userIds) {
if (CollectionUtils.isEmpty(userIds)) {
return;
}
redisTemplate.executePipelined(new DeleteListCallBack(userIds, updateKey));
}
@AllArgsConstructor
private static class DeleteListCallBack implements SessionCallback<Object> {
private Set<Integer> userIds;
private String updateKey;
@Override
public <K, V> Object execute(RedisOperations<K, V> operations) throws DataAccessException {
ListOperations<String, String> listOperations = (ListOperations<String, String>) operations.opsForList();
userIds.forEach(userId -> listOperations.remove(updateKey, 1, userId.toString()));
return null;
}
}在數(shù)據(jù)量比較小的時(shí)候沒(méi)有出現(xiàn)問(wèn)題，直到有一條收到了Redis的內(nèi)存和cpu利用率的告警消息，我們發(fā)現(xiàn)這么使用是有問(wèn)題的，核心原因在于list的lrem操作的時(shí)間復(fù)雜度是O(N+M)，其中N是list的長(zhǎng)度, M是要移除的元素的個(gè)數(shù)，而我們這里還是一個(gè)一個(gè)移除的，當(dāng)然會(huì)導(dǎo)致Redis數(shù)據(jù)積壓和cpu每秒ops升高導(dǎo)致cpu利用率飚高。也就是說(shuō)，即使使用Pipeline進(jìn)行批量操作，但是由于單次操作很耗時(shí)，是會(huì)導(dǎo)致整個(gè)Redis出現(xiàn)問(wèn)題的。
后面我們進(jìn)行了優(yōu)化，選用了list的ltrim命令，一次命令執(zhí)行批量remove操作：
public void deleteSet(String updateKey, Set<Integer> deviceIds) {
if (CollectionUtils.isEmpty(deviceIds)) {
return;
}
int maxSize = 10000;
redisTemplate.opsForList().trim(updateKey, maxSize + 1, -1);
}由于ltrim本身的時(shí)間復(fù)雜度為O(M), 其中M要移除的元素的個(gè)數(shù)，相比于原始方案的lrem，效率提升很多，可以不需要使用Redis Pipeline，優(yōu)化結(jié)果使得Redis內(nèi)存利用率和cpu利用率都極大程度得到緩解。
六、Redisson 對(duì) Redis Pipeline 特性支持
在redisson官方文檔中額外特性介紹中有說(shuō)到批量命令執(zhí)行這個(gè)特性， 也就是多個(gè)命令在一次網(wǎng)絡(luò)調(diào)用中集中發(fā)送，該特性是RBatch這個(gè)類(lèi)支持的，從這個(gè)類(lèi)的描述來(lái)看，主要是為Redis Pipeline這個(gè)特性服務(wù)的，并且主要是通過(guò)隊(duì)列和異步實(shí)現(xiàn)的。
/**
* Interface for using Redis Pipeline feature.
* <p>
* All method invocations on objects got through this interface
* are batched to separate queue and could be executed later
* with <code>execute()</code> or <code>executeAsync()</code> methods.
*
*
* @author Nikita Koksharov
*
*/
public interface RBatch {
/**
* Returns stream instance by <code>name</code>
*
* @param <K> type of key
* @param <V> type of value
* @param name of stream
* @return RStream object
*/
<K, V> RStreamAsync<K, V> getStream(String name);
/**
* Returns stream instance by <code>name</code>
* using provided <code>codec</code> for entries.
*
* @param <K> type of key
* @param <V> type of value
* @param name - name of stream
* @param codec - codec for entry
* @return RStream object
*/
<K, V> RStreamAsync<K, V> getStream(String name, Codec codec);
......
/**
* Returns list instance by name.
*
* @param <V> type of object
* @param name - name of object
* @return List object
*/
<V> RListAsync<V> getList(String name);
<V> RListAsync<V> getList(String name, Codec codec);
......
/**
* Executes all operations accumulated during async methods invocations.
* <p>
* If cluster configuration used then operations are grouped by slot ids
* and may be executed on different servers. Thus command execution order could be changed
*
* @return List with result object for each command
* @throws RedisException in case of any error
*
*/
BatchResult<?> execute() throws RedisException;
/**
* Executes all operations accumulated during async methods invocations asynchronously.
* <p>
* In cluster configurations operations grouped by slot ids
* so may be executed on different servers. Thus command execution order could be changed
*
* @return List with result object for each command
*/
RFuture<BatchResult<?>> executeAsync();
/**
* Discard batched commands and release allocated buffers used for parameters encoding.
*/
void discard();
/**
* Discard batched commands and release allocated buffers used for parameters encoding.
*
* @return void
*/
RFuture<Void> discardAsync();
}簡(jiǎn)單的測(cè)試代碼如下：
@Slf4j
public class RedisPipelineTest {
public static void main(String[] args) {
//Redisson使用Pipeline模式
Config config = new Config();
config.useSingleServer().setAddress("redis://xx.xx.xx.xx:6379");
RedissonClient redisson = Redisson.create(config);
RBatch redisBatch = redisson.createBatch();
int size = 100000;
String zSetKey = "Pipeline-test-set";
long begin = System.currentTimeMillis();
//將命令放入隊(duì)列中
for (int i = 0; i < size; i++) {
redisBatch.getSet(zSetKey + i).addAsync("ccc");
}
//批量執(zhí)行命令
redisBatch.execute();
log.info("Redisson Pipeline模式耗時(shí)：{}ms", (System.currentTimeMillis() - begin));
//關(guān)閉
redisson.shutdown();
}
}核心方法分析：
1.建Redisson客戶(hù)端RedissonClient redisson = redisson.create(config), 該方法最終會(huì)調(diào)用Reddison的構(gòu)造方法Redisson(Config config)。
protected Redisson(Config config) {
this.config = config;
Config configCopy = new Config(config);
connectionManager = ConfigSupport.createConnectionManager(configCopy);
RedissonObjectBuilder objectBuilder = null;
if (config.isReferenceEnabled()) {
objectBuilder = new RedissonObjectBuilder(this);
}
//新建異步命令執(zhí)行器
commandExecutor = new CommandSyncService(connectionManager, objectBuilder);
//執(zhí)行刪除超時(shí)任務(wù)的定時(shí)器
evictionScheduler = new EvictionScheduler(commandExecutor);
writeBehindService = new WriteBehindService(commandExecutor);
}該構(gòu)造方法中會(huì)新建異步命名執(zhí)行器CommandAsyncExecutor commandExecutor和用戶(hù)刪除超時(shí)任務(wù)的EvictionScheduler evictionScheduler。
2.創(chuàng)建RBatch實(shí)例RBatch redisBatch = redisson.createBatch()， 該方法會(huì)使用到步驟1中的commandExecutor和evictionScheduler實(shí)例對(duì)象。
@Override
public RBatch createBatch(BatchOptions options) {
return new RedissonBatch(evictionScheduler, commandExecutor, options);
}
public RedissonBatch(EvictionScheduler evictionScheduler, CommandAsyncExecutor executor, BatchOptions options) {
this.executorService = new CommandBatchService(executor, options);
this.evictionScheduler = evictionScheduler;
}其中的options對(duì)象會(huì)影響后面批量執(zhí)行命令的流程。
3. 異步給set集合添加元素的操作addAsync，這里會(huì)具體調(diào)用RedissonSet的addAsync方法
@Override
public RFuture<Boolean> addAsync(V e) {
String name = getRawName(e);
return commandExecutor.writeAsync(name, codec, RedisCommands.SADD_SINGLE, name, encode(e));
}（1）接著調(diào)用CommandAsyncExecutor的異步寫(xiě)入方法writeAsync。
@Override
public <T, R> RFuture<R> writeAsync(String key, Codec codec, RedisCommand<T> command, Object... params) {
RPromise<R> mainPromise = createPromise();
NodeSource source = getNodeSource(key);
async(false, source, codec, command, params, mainPromise, false);
return mainPromise;
}（2） 接著調(diào)用批量命令執(zhí)行器
CommandBatchService的異步發(fā)送命令。
@Override
public <V, R> void async(boolean readOnlyMode, NodeSource nodeSource,
Codec codec, RedisCommand<V> command, Object[] params, RPromise<R> mainPromise, boolean ignoreRedirect) {
if (isRedisBasedQueue()) {
boolean isReadOnly = options.getExecutionMode() == ExecutionMode.REDIS_READ_ATOMIC;
RedisExecutor<V, R> executor = new RedisQueuedBatchExecutor<>(isReadOnly, nodeSource, codec, command, params, mainPromise,
false, connectionManager, objectBuilder, commands, connections, options, index, executed, latch, referenceType);
executor.execute();
} else {
//執(zhí)行分支
RedisExecutor<V, R> executor = new RedisBatchExecutor<>(readOnlyMode, nodeSource, codec, command, params, mainPromise,
false, connectionManager, objectBuilder, commands, options, index, executed, referenceType);
executor.execute();
}
}（3） 接著調(diào)用了RedisBatchExecutor.
execute方法和BaseRedisBatchExecutor.
addBatchCommandData方法。
@Override
public void execute() {
addBatchCommandData(params);
}
protected final void addBatchCommandData(Object[] batchParams) {
MasterSlaveEntry msEntry = getEntry(source);
Entry entry = commands.get(msEntry);
if (entry == null) {
entry = new Entry();
Entry oldEntry = commands.putIfAbsent(msEntry, entry);
if (oldEntry != null) {
entry = oldEntry;
}
}
if (!readOnlyMode) {
entry.setReadOnlyMode(false);
}
Codec codecToUse = getCodec(codec);
BatchCommandData<V, R> commandData = new BatchCommandData<V, R>(mainPromise, codecToUse, command, batchParams, index.incrementAndGet());
entry.getCommands().add(commandData);
}這里的commands以主節(jié)點(diǎn)為KEY,以待發(fā)送命令隊(duì)列列表為VALUE(Entry)，保存一個(gè)MAP.然后會(huì)把命令都添加到entry的commands命令隊(duì)列中， Entry結(jié)構(gòu)如下面代碼所示。
public static class Entry {
Deque<BatchCommandData<?, ?>> commands = new LinkedBlockingDeque<>();
volatile boolean readOnlyMode = true;
public Deque<BatchCommandData<?, ?>> getCommands() {
return commands;
}
public void setReadOnlyMode(boolean readOnlyMode) {
this.readOnlyMode = readOnlyMode;
}
public boolean isReadOnlyMode() {
return readOnlyMode;
}
public void clearErrors() {
for (BatchCommandData<?, ?> commandEntry : commands) {
commandEntry.clearError();
}
}
}4. 批量執(zhí)行命令redisBatch.execute()，這里會(huì)最終調(diào)用CommandBatchService的executeAsync方法，該方法完整代碼如下，我們下面來(lái)逐一進(jìn)行拆解。
public RFuture<BatchResult<?>> executeAsync() {
......
RPromise<BatchResult<?>> promise = new RedissonPromise<>();
RPromise<Void> voidPromise = new RedissonPromise<Void>();
if (this.options.isSkipResult()
&& this.options.getSyncSlaves() == 0) {
......
} else {
//這里是對(duì)異步執(zhí)行結(jié)果進(jìn)行處理,可以先忽略, 后面會(huì)詳細(xì)講，先關(guān)注批量執(zhí)行命令的邏輯
voidPromise.onComplete((res, ex) -> {
......
});
}
AtomicInteger slots = new AtomicInteger(commands.size());
......
//真正執(zhí)行的代碼入口，批量執(zhí)行命令
for (Map.Entry<MasterSlaveEntry, Entry> e : commands.entrySet()) {
RedisCommonBatchExecutor executor = new RedisCommonBatchExecutor(new NodeSource(e.getKey()), voidPromise,
connectionManager, this.options, e.getValue(), slots, referenceType);
executor.execute();
}
return promise;
}里面會(huì)用到我們?cè)?.3步驟所生成的commands實(shí)例。
（1）接著調(diào)用了基類(lèi)RedisExecutor的execute方法
public void execute() {
......
connectionFuture.onComplete((connection, e) -> {
if (connectionFuture.isCancelled()) {
connectionManager.getShutdownLatch().release();
return;
}
if (!connectionFuture.isSuccess()) {
connectionManager.getShutdownLatch().release();
exception = convertException(connectionFuture);
return;
}
//調(diào)用RedisCommonBatchExecutor的sendCommand方法, 里面會(huì)將多個(gè)命令放到一個(gè)List<CommandData<?, ?>> list列表里面
sendCommand(attemptPromise, connection);
writeFuture.addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture future) throws Exception {
checkWriteFuture(writeFuture, attemptPromise, connection);
}
});
});
......
}（2）接著調(diào)用
RedisCommonBatchExecutor的sendCommand方法，里面會(huì)將多個(gè)命令放到一個(gè)List<commanddata> list列表里面。
@Override
protected void sendCommand(RPromise<Void> attemptPromise, RedisConnection connection) {
boolean isAtomic = options.getExecutionMode() != ExecutionMode.IN_MEMORY;
boolean isQueued = options.getExecutionMode() == ExecutionMode.REDIS_READ_ATOMIC
|| options.getExecutionMode() == ExecutionMode.REDIS_WRITE_ATOMIC;
//將多個(gè)命令放到一個(gè)List<CommandData<?, ?>> list列表里面
List<CommandData<?, ?>> list = new ArrayList<>(entry.getCommands().size());
if (source.getRedirect() == Redirect.ASK) {
RPromise<Void> promise = new RedissonPromise<Void>();
list.add(new CommandData<Void, Void>(promise, StringCodec.INSTANCE, RedisCommands.ASKING, new Object[] {}));
}
for (CommandData<?, ?> c : entry.getCommands()) {
if ((c.getPromise().isCancelled() || c.getPromise().isSuccess())
&& !isWaitCommand(c)
&& !isAtomic) {
// skip command
continue;
}
list.add(c);
}
......
//調(diào)用RedisConnection的send方法，將命令一次性發(fā)到Redis服務(wù)器端
writeFuture = connection.send(new CommandsData(attemptPromise, list, options.isSkipResult(), isAtomic, isQueued, options.getSyncSlaves() > 0));
}（3）接著調(diào)用RedisConnection的send方法，通過(guò)Netty通信發(fā)送命令到Redis服務(wù)器端執(zhí)行，這里也驗(yàn)證了Redisson客戶(hù)端底層是采用Netty進(jìn)行通信的。
public ChannelFuture send(CommandsData data) {
return channel.writeAndFlush(data);
}5. 接收返回結(jié)果，這里主要是監(jiān)聽(tīng)事件是否完成，然后組裝返回結(jié)果, 核心方法是步驟4提到的CommandBatchService的executeAsync方法,里面會(huì)對(duì)返回結(jié)果進(jìn)行監(jiān)聽(tīng)和處理, 核心代碼如下：
public RFuture<BatchResult<?>> executeAsync() {
......
RPromise<BatchResult<?>> promise = new RedissonPromise<>();
RPromise<Void> voidPromise = new RedissonPromise<Void>();
if (this.options.isSkipResult()
&& this.options.getSyncSlaves() == 0) {
......
} else {
voidPromise.onComplete((res, ex) -> {
//對(duì)返回結(jié)果的處理
executed.set(true);
......
List<Object> responses = new ArrayList<Object>(entries.size());
int syncedSlaves = 0;
for (BatchCommandData<?, ?> commandEntry : entries) {
if (isWaitCommand(commandEntry)) {
syncedSlaves = (Integer) commandEntry.getPromise().getNow();
} else if (!commandEntry.getCommand().getName().equals(RedisCommands.MULTI.getName())
&& !commandEntry.getCommand().getName().equals(RedisCommands.EXEC.getName())
&& !this.options.isSkipResult()) {
......
//獲取單個(gè)命令的執(zhí)行結(jié)果
Object entryResult = commandEntry.getPromise().getNow();
......
//將單個(gè)命令執(zhí)行結(jié)果放到List中
responses.add(entryResult);
}
}
BatchResult<Object> result = new BatchResult<Object>(responses, syncedSlaves);
promise.trySuccess(result);
......
});
}
......
return promise;
}這里會(huì)把單個(gè)命令的執(zhí)行結(jié)果放到responses里面，最終返回RPromise<batchresult>promise。
從上面的分析來(lái)看，Redisson客戶(hù)端對(duì)Redis Pipeline的支持也是從多個(gè)命令在一次網(wǎng)絡(luò)通信中執(zhí)行和異步處理來(lái)實(shí)現(xiàn)的。
七、總結(jié)
Redis提供了Pipelining進(jìn)行批量操作的高級(jí)特性，極大地提高了部分?jǐn)?shù)據(jù)類(lèi)型沒(méi)有批量執(zhí)行命令導(dǎo)致的執(zhí)行耗時(shí)而引起的性能問(wèn)題，但是我們?cè)谑褂玫倪^(guò)程中需要考慮Pipeline操作中單個(gè)命令執(zhí)行的耗時(shí)問(wèn)題，否則帶來(lái)的效果可能適得其反。最后擴(kuò)展分析了Redisson客戶(hù)端對(duì)Redis Pipeline特性的支持原理，可以與Lettuce客戶(hù)端對(duì)Redis Pipeline支持原理進(jìn)行比較，加深Pipeline在不同Redis客戶(hù)端實(shí)現(xiàn)方式的理解。
