How to free up disk space on an Ubuntu Nginx or Apache Web Server
Last updated: November 22nd 2022
Wondering why your webserver is running out of disk space despite not having uploaded any files to it? In this article we go through the most common methods of discovering what is using your disk space, and what you can remove safely in order to free up some disk space.
This guide assumes you are on Ubuntu 16.04 Xenial or 18.04 Bionic or later. Most of the commands listed here will work with other Linux distributions as well.
In addition, you should have shell (SSH) access to your VPS.
Discovering what is consuming disk space
You can start with checking out the overall disk space consumption for your system. You do this by executing the df command. On a typical Webdock VPS this will look like:
# sudo df -h
Filesystem Size Used Avail Use% Mounted on lxd/containers/maintenance 94G 2.6G 91G 3% / none 492K 4.0K 488K 1% /dev udev 189G 0 189G 0% /dev/tty tmpfs 100K 0 100K 0% /dev/lxd tmpfs 100K 0 100K 0% /dev/.lxd-mounts tmpfs 189G 0 189G 0% /dev/shm tmpfs 189G 192K 189G 1% /run tmpfs 5.0M 0 5.0M 0% /run/lock tmpfs 189G 0 189G 0% /sys/fs/cgroup
The -h argument we pass to df just means "human readable" so you get the disk space utilization in Megabytes, Gigabytes etc. instead of bytes which can be hard to read.
Here you can see our server is consuming 2.6 Gigabyte of a total of 94 Gigabyte available. Usually the partition you are interested in, is the one mounted on / this is the root of your system and which (usually) holds all your data and applications.
But where is all that space being used? Here we use the du, or disk utilization, command.
# sudo du -h --max-depth=1 / 2>/dev/null
512 /mnt 43K /tmp 0 /sys 30K /home 512 /opt 512 /srv 4.0K /dev 874M /var 17M /sbin 512 /boot 1.7G /usr 18M /bin 8.3M /etc 192K /run 4.0K /proc 184K /root 1.0K /lib64 512 /media 5.0K /snap 52M /lib 2.6G /
This command may take a while to run as it is iterating throughout your entire filesystem - all files and folders - and tallying up disk utilization.
As you can see from the highlighted entries /var and /usr is using up the most disk space. This is normal for an Ubuntu webserver, as in /var you have all your data typically (website/application and database data) and under /usr you have all your installed applications.
But what specifically in /var is using up space? Here you just modify the path you pass to the du command to see which directories under /var are using the most disk space:
Tip: Instead of using the "du" you can also "apt install ncdu" and then run it with "ncdu" which is a graphical equivalent to du, enabling you to browse your filesystem in a, possibly easier, interface than running du commands over and over as shown here below.
# sudo du -h --max-depth=1 /var 2>/dev/null
80M /var/cache 13K /var/spool 512 /var/snap 587M /var/lib 1.9M /var/backups 171M /var/www 512 /var/opt 28K /var/tmp 512 /var/crash 512 /var/local 512 /var/mail 35M /var/log 874M /var
The directories you need to pay attention to here are your /var/log and /var/www directories. It is normal for your website to use a lot of space, but you may want to dig deeper there and check if your /var/www/logs/ (on Webdock servers at least) is consuming a lot of space. This would indicate that either your access log is very big as you get a lot of traffic (congratulations!) or your error log is very big, which might indicate a problem with your application.
For example if your website is throwing lots of PHP warnings, you may want to fix those issues.
If your access log is very big, but your website is a low traffic website - you may want to inspect the log to see what is happening. Maybe you are being subjected to brute force or DOS attackes which generates a lot of activity in your logs.
If your /var/log directory is consuming lots of space, you should also inspect the contents of that directory and try to pin down which software is eating up lots of log space. There are a few ways to clean up logs which we will go into in the next section.
Useful commands for freeing up disk space
Once you've discovered what the cause is of your disk space consumption you can much more easily figure out what to do about it. But regardless of what is using up your disk space, here are some commands you can safely run in order to free up disk space on any Webdock webserver.
Changing the Docker storage driver
You can save a lot of disk space and deploy your Docker images faster on Webdock if you change to the fuse-overlayfs storage driver instead of the vfs default. Click here to read our guide on how to change to fuse-overlayfs on existing Docker installations.
Clearing Docker Image and Build Cache
If you are running Docker on your server, it may be helpful to clear the Docker Build and Image Caches:
# sudo docker builder prune # sudo docker image prune
In some cases you will see the docker image directory - /var/lib/docker/vfs - bloat up on VFS backed servers. This is apparently expected behavior and is due to docker keeping a lot of duplicate data around. As far as we are aware, there is currently no way to mitigate this additional bloat at this time.
Cleaning log files
There are a few commands you can run to clean out extraneous log files:
# sudo journalctl --vacuum-size=1M
This command essentially cleans out your journal logs and reduces them down to 1 Megabyte. This log can over time grow quite big.
# sudo rm /var/log/*.gz
This command removes any old, gzipped, files in your log directory which logrotate hasn't deleted on its own yet. These files will automatically be removed over time, but they are safe to delete if you need some disk space right away.
Permanently limiting the size of your Journal (Systemd) logs
If you want to set an upper limit on your journal logs so they never grow too large, you can do so by editing /etc/systemd/journald.conf in your favorite editor:
# sudo nano /etc/systemd/journald.conf
Look for the SystemMaxUse directive, uncomment it and set it to something like 50M or whatever you want:
... #RateLimitBurst=1000 SystemMaxUse=50M #SystemKeepFree= ...
As you can see, there are lots of other knobs and settings un there you can tweak. Check out the man page for journald for details. Once finished, restart the journald service:
# sudo systemctl restart systemd-journald
Clearing the Composer cache
If you use Composer you may want to clear the cache as it can grow quite big over time.
# sudo composer clearcache
Cleaning out your apt cache and unneeded packages
Apt can sometimes keep some junk around which is no longer needed. In order to clean this up you can run the following commands
# sudo apt clean; sudo apt autoremove --purge;
Uninstalling packages you are not using
In a typical Webdock stack we pre-install stuff like MongoDB and Redis. You should take a look at our Perfect Server guides in order to see everything which has been pre-installed and if there is stuff you are not using, then you can safely remove it. Let's assume you are just running a basic Wordpress site that just needs the webserver and MySQL - you can then run the following to free up quite a bit of disk space
# sudo apt remove -y --purge lftp bzip2 arj nomarch lzop gcc binutils make zsh nodejs redis-server libcurl4-openssl-dev pkg-config libssl-dev mongodb # sudo apt autoremove -y
And to remove the MongoDB PHP extension
# sudo pecl uninstall mongodb
This will typically (at the time of writing) free up about 500Mb of disk space
Disabling the MariaDB Binary Log
WARNING: Disabling the binary log may be dangerous in case your server unexpectedly crashes as this will prevent MariaDB from reconstructing data which had not been written to the database yet. Only disable the binary log if you really have to.
If you notice that the MySQl log is very big in /var/log/mysql - you have a lot of big .bin files in there then this is your MariaDB binary log which has grown large. You should first of all inspect this log to see if your application has some errors which is throwing a lot of noise. If it's just due to your site having a lot of traffic, you may want to consider disabling this log. You do so by editing your MariaDB config:
# sudo nano /etc/mysql/my.cnf
And add skip-log-bin in the [mysqld] section
Further down you should also comment out the following lines if they are enabled:
#log_bin = /var/log/mysql/mariadb-bin #log_bin_index = /var/log/mysql/mariadb-bin.index
And then you restart MariaDB
# sudo systemctl restart mariadb
Now it is safe to delete all those .bin files in the mysql log directory
It is always a good idea to check out why your webserver is consuming a lot more disk space than you expect as this can reveal problems with your application or attacks/hacks.
Running all of these commands on a basic Webdock webserver will save you approximately 500Mb of disk space. Most of this is due to uninstalling software, as we always clean our base images as much as possible before publishing.
We hope this guide helps you in optimizing your server and getting the most out of the resources you are paying for. If all else fails, it is fortunately easy to scale your VPS to your needs.