Prerequisites

This guide assumes you are on Ubuntu 16.04 Xenial or 18.04 Bionic or later. Most of the commands listed here will work with other Linux distributions as well.

In addition, you should have shell (SSH) access to your VPS.

Discovering what is consuming disk space

You can start with checking out the overall disk space consumption for your system. You do this by executing the df command. On a typical Webdock VPS this will look like:

# sudo df -h

Sample output:

Filesystem                  Size  Used Avail Use% Mounted on
lxd/containers/maintenance   94G  2.6G   91G   3% /
none                        492K  4.0K  488K   1% /dev
udev                        189G     0  189G   0% /dev/tty
tmpfs                       100K     0  100K   0% /dev/lxd
tmpfs                       100K     0  100K   0% /dev/.lxd-mounts
tmpfs                       189G     0  189G   0% /dev/shm
tmpfs                       189G  192K  189G   1% /run
tmpfs                       5.0M     0  5.0M   0% /run/lock
tmpfs                       189G     0  189G   0% /sys/fs/cgroup

The -h argument we pass to df just means "human readable" so you get the disk space utilization in Megabytes, Gigabytes etc. instead of bytes which can be hard to read.

Here you can see our server is consuming 2.6 Gigabyte of a total of 94 Gigabyte available. Usually the partition you are interested in, is the one mounted on / this is the root of your system and which (usually) holds all your data and applications.

But where is all that space being used? Here we use the du, or disk utilization, command.

# sudo du -h --max-depth=1 / 2>/dev/null

Sample output:

512     /mnt
43K     /tmp
0       /sys
30K     /home
512     /opt
512     /srv
4.0K    /dev
874M    /var
17M     /sbin
512     /boot
1.7G    /usr
18M     /bin
8.3M    /etc
192K    /run
4.0K    /proc
184K    /root
1.0K    /lib64
512     /media
5.0K    /snap
52M     /lib
2.6G    /

This command may take a while to run as it is iterating throughout your entire filesystem - all files and folders - and tallying up disk utilization.

As you can see from the highlighted entries /var and /usr is using up the most disk space. This is normal for an Ubuntu webserver, as in /var you have all your data typically (website/application and database data) and under /usr you have all your installed applications.

But what specifically in /var is using up space? Here you just modify the path you pass to the du command to see which directories under /var are using the most disk space:

Tip: Instead of using the "du" you can also "apt install ncdu" and then run it with "ncdu" which is a graphical equivalent to du, enabling you to browse your filesystem in a, possibly easier, interface than running du commands over and over as shown here below.

# sudo du -h --max-depth=1 /var 2>/dev/null

Sample output:

80M     /var/cache
13K     /var/spool
512     /var/snap
587M    /var/lib
1.9M    /var/backups
171M    /var/www
512     /var/opt
28K     /var/tmp
512     /var/crash
512     /var/local
512     /var/mail
35M     /var/log
874M    /var

The directories you need to pay attention to here are your /var/log and /var/www directories. It is normal for your website to use a lot of space, but you may want to dig deeper there and check if your /var/www/logs/ (on Webdock servers at least)  is consuming a lot of space. This would indicate that either your access log is very big as you get a lot of traffic (congratulations!) or your error log is very big, which might indicate a problem with your application.

For example if your website is throwing lots of PHP warnings, you may want to fix those issues.

If your access log is very big, but your website is a low traffic website - you may want to inspect the log to see what is happening. Maybe you are being subjected to brute force or DOS attackes which generates a lot of activity in your logs.

If your /var/log directory is consuming lots of space, you should also inspect the contents of that directory and try to pin down which software is eating up lots of log space. There are a few ways to clean up logs which we will go into in the next section.

Useful commands for freeing up disk space

Once you've discovered what the cause is of your disk space consumption you can much more easily figure out what to do about it. But regardless of what is using up your disk space, here are some commands you can safely run in order to free up disk space on any Webdock webserver.

Changing the Docker storage driver

You can save a lot of disk space and deploy your Docker images faster on Webdock if you change to the fuse-overlayfs storage driver instead of the vfs default. Click here to read our guide on how to change to fuse-overlayfs on existing Docker installations.

Clearing Docker Image and Build Cache

If you are running Docker on your server, it may be helpful to clear the Docker Build and Image Caches:

# sudo docker builder prune
# sudo docker image prune

In some cases you will see the docker image directory - /var/lib/docker/vfs - bloat up on VFS backed servers. This is apparently expected behavior and is due to docker keeping a lot of duplicate data around. As far as we are aware, there is currently no way to mitigate this additional bloat at this time.

Cleaning log files

There are a few commands you can run to clean out extraneous log files:

# sudo journalctl --vacuum-size=1M

This command essentially cleans out your journal logs and reduces them down to 1 Megabyte. This log can over time grow quite big.

# sudo rm /var/log/*.gz

This command removes any old, gzipped, files in your log directory which logrotate hasn't deleted on its own yet. These files will automatically be removed over time, but they are safe to delete if you need some disk space right away.

Permanently limiting the size of your Journal (Systemd) logs

If you want to set an upper limit on your journal logs so they never grow too large, you can do so by editing /etc/systemd/journald.conf in your favorite editor:

# sudo nano /etc/systemd/journald.conf

Look for the SystemMaxUse directive, uncomment it and set it to something like 50M or whatever you want:

...
#RateLimitBurst=1000
SystemMaxUse=50M
#SystemKeepFree=
...

As you can see, there are lots of other knobs and settings un there you can tweak. Check out the man page for journald for details. Once finished, restart the journald service:

# sudo systemctl restart systemd-journald

Clearing the Composer cache

If you use Composer you may want to clear the cache as it can grow quite big over time.

# sudo composer clearcache

Cleaning out your apt cache and unneeded packages

Apt can sometimes keep some junk around which is no longer needed. In order to clean this up you can run the following commands

# sudo apt clean; sudo apt autoremove --purge;

Uninstalling packages you are not using

In a typical Webdock stack we pre-install stuff like MongoDB and Redis. You should take a look at our Perfect Server guides in order to see everything which has been pre-installed and if there is stuff you are not using, then you can safely remove it. Let's assume you are just running a basic Wordpress site that just needs the webserver and MySQL - you can then run the following to free up quite a bit of disk space

# sudo apt remove -y --purge lftp bzip2 arj nomarch lzop gcc binutils make zsh nodejs redis-server libcurl4-openssl-dev pkg-config libssl-dev mongodb
# sudo apt autoremove -y

And to remove the MongoDB PHP extension

# sudo pecl uninstall mongodb

This will typically (at the time of writing) free up about 500Mb of disk space

Disabling the MariaDB Binary Log

WARNING: Disabling the binary log may be dangerous in case your server unexpectedly crashes as this will prevent MariaDB from reconstructing data which had not been written to the database yet. Only disable the binary log if you really have to.

If you notice that the MySQl log is very big in /var/log/mysql - you have a lot of big .bin files in there then this is your MariaDB binary log which has grown large. You should first of all inspect this log to see if your application has some errors which is throwing a lot of noise. If it's just due to your site having a lot of traffic, you may want to consider disabling this log. You do so by editing your MariaDB config:

$ sudo nano /etc/mysql/my.cnf

OR

$ sudo nano /etc/mysql/mariadb.conf.d/50-server.cnf

And add skip-log-bin in the [mysqld] section

[mysqld]
skip-log-bin

Further down you should also comment out the following lines if they are enabled:

#log_bin                = /var/log/mysql/mariadb-bin
#log_bin_index          = /var/log/mysql/mariadb-bin.index

And then you restart MariaDB

# sudo systemctl restart mariadb

Now it is safe to delete all those .bin files in the mysql log directory

Conclusion

It is always a good idea to check out why your webserver is consuming a lot more disk space than you expect as this can reveal problems with your application or attacks/hacks.

Running all of these commands on a basic Webdock webserver will save you approximately 500Mb of disk space. Most of this is due to uninstalling software, as we always clean our base images as much as possible before publishing.

We hope this guide helps you in optimizing your server and getting the most out of the resources you are paying for. If all else fails, it is fortunately easy to scale your VPS to your needs.