Recently I’ve been putting time into making improvements to the server infrastructure for Holiday API.
This has included making things more fault tolerant and building out a new cluster of web servers. The biggest improvement was finally adding a continuous deployment pipeline to complement the existing continuous integration flow.
Everything has come together quite well, and I was even able to flip the
switch this past weekend, without an outage.
That’s not to say things came together as smoothly as I would have liked. This
has been something I’ve been chipping away at for the last week or two, making adjustments and tuning along the way.
Deployment problems with symbolic links
All of a sudden, I had a new issue with regard to how the code was being linked on the server. Originally I was deploying to a single directory, no symbolic
linking (or symlinking) involved at all.
With the new flow, I opted to build the release and then symlink to the “current” release. This was all then symlinked to a directory in /var/www/
to marry up with how the existing Docker configuration works.
Nothing revolutionary or anything and everything actually worked great. When the pipeline ran, new code would go live and all was well.
That was, until I made a change to the primary index.php file in the document
root. This particular file’s changes would not register until I restarted thephp-fpm service.
Changes to other PHP files were picked up immediately. The issue was limited to the aforementioned index.php file in the document root.
The old web server didn’t have this issue, so I knew it had something to do with
the new server’s configuration. Without knowing for certain, I assumed it had
something to do with the symlinks that were now in play.
At this point, I did quite a bit of web searching, and ran into quite a few web
sites that were discussing getting PHP files to serve correctly with nginx, and
not necessarily anything related to my issue.
Finally, I happened upon a Stack Overflow post that seemed like what I was
experiencing, but sadly was scummed up by some asshole letting the original
poster know that their issue wasn’t well researched at all and they should not have even posted the question.
Document root vs the real-time path root
This was my first time running into the issue, and at least for me and my
Google Fu, it was a pain in the butt to find a solution for the issue.
Eventually, after more research, going through the nginx documentation and some trial and error, I did figure things out. It had to do with the php-fpm portion of my nginx configuration file.
Before, the configuration for my server, looked a little something like this
(abridged for the sake of this post):
server {
location ~ \.php$ {
try_files $uri =404;
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/run/php/php-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
}
}Turns out, the big issue here is the $document_root variable being used in the fastcgi_param.
As I came to find out, when using symbolic links, that particular variable ends
up being converted to an absolute path and is effectively cached when the server starts or something like that.
Fortunately, there’s another variable that doesn’t does fall victim of this
alleged caching. The $realpath_root variable is a drop in replacement for the $document_root.
Eventually, the updated file looked more like this:
server {
location ~ \.php$ {
try_files $uri =404;
include /etc/nginx/fastcgi_params;
fastcgi_pass unix:/run/php/php-fpm.sock;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
}
}After quick restart of both nginx and php-fpm, I went ahead and deployed some new code that affected the index.php in the document root. Afterwards, the changes were showing up and all was well again!
