Array chunking is a great way to paginate large arrays and as I found out recently, makes it very easy to map a large array from Redis (by way of mget and/or pipelining) back to the original data. Chunking an array is the process of taking an array and splitting it into chunks. This can be done easily with the array_chunk() function.
Let’s say you have large array of 100 records and you want to break it into groups of 10, to do so you would feed it to array_chunk() with the value of 10:
$array = array_fill(0, 99, 'testing'
$chunks = array_chunk($array, 10
echo count($chunks // Will be 10
print_r($chunks // So you can see it ;)
From time to time I will keep a full data set in cache and use this method to be able to break the data out into pages. Once it’s chunked, you can request it as if it were a page of data:
$page = 2
print_r($chunks[$page - 1
We have to decrease the requested page number by one because the chunked array will start at 0.
The other use case that I’ve utilized the chunking with is when you use mget to retrieve multiple keys from Redis or when utilizing pipelining with Redis both of which will result in one flat array. Let’s say we have an array of users and want to loop through and pull 3 keys each out of Redis for each user. You could loop through the users and run mget for each user, but this is obviously inefficient and you should attempt to pull all of the keys at once or better still, pipeline it to reduce the network overhead. I’ll show both examples as they can both benefit from array chunking.
First, using mget to pull multiple keys for the users:
$user_keys = array
// Loops through the users
foreach ($user_ids as $user_id)
{
// Creates a key prefix for this user
$key_prefix = 'user:' . $user_id . ':'
// Adds each key to the array
foreach (array('name', 'posts', 'location') as $key)
{
$user_keys[] = $key_prefix . $key
}
}
// Uses the array to pull all of the data
// returns count($user_ids) * 3 values in a single array
$user_values = array_chunk($redis->mget($user_keys), 3
$users = array
// Loops through the user IDs again to map the data back
foreach ($user_ids as $user_key => $user_id)
{
$users[] = array(
'id' => $user_id,
'name' => $user_values[$user_key][0
'posts' => $user_values[$user_key][1
'location' => $user_values[$user_key][2
}
The Redis command mget returns a flat array of all of the values (null if they aren’t present, maintaining a full structure), once you chunk the data you could use array_combine() to apply key names to the chucked array instead of using the integer keys as this example shows. I’ll show you that in the next example 😉
The next example works around Redis pipelining, which would work if you are mixing Redis commands like get and zcard and such. Using the same hypothetical user IDs array from earlier:
$pipe = $redis->pipeline
// Loops through the users and pulls their username, total updates count and the last 10 updates
foreach ($user_ids as $user_id)
{
$key_prefix = 'user:' . $user_id . ':'
$pipe->get($key_prefix . 'username'
$pipe->zcard($key_prefix . 'count:updates'
$pipe->zrevrange($key_prefix . 'updates', 0, 9
}
// Pulls the data just like before and chunks it out
$user_values = array_chunk($pipe->execute(), 3
$users = array
$user_keys = array('username', 'updates_total', 'updates_last10'
// Loops through the user IDs again to map the data back
foreach ($user_ids as $user_key => $user_id)
{
$user = array_combine($user_key, $user_values[$user_key
$user['id'] = $user_id
$users[] = $user
}
Similar workflow to the previous example but actually a bit simpler with the inclusion of the array_combine() and could probably be further simplified by using the $user_id value as the key on the $users array instead of including it in as an array element. Thinking it through further, because the Redis information is returned in the other that you request it, you could use the $user_ids array and the chunks return from Redis with array_combine to reassemble the data in a single line:
$users = array_combine($user_ids, $user_values // BOOM!
So that’s how you can use array_chunk() along with some of my own use cases. If you already use array_chunk() or can think of another use case that I didn’t touch on, comment below!
