Your server must support mods.
You can install mods on Forge servers and modpacks.
You can't install mods on Spigot, Bukkit, Standard Minecraft.

To install mods you will need to get .jar files.
Search and download Minecraft Server mods here:
https://minecraft.curseforge.com/mc-mods
https://mods.curse.com/mc-mods/minecraft

As an example, we will describe the Iron Chests mod installation process: https://minecraft.curseforge.com/projects/iron-chests.

In order to upload your mods go to the Your Gameserver → File manager page, open folder "mods" and click Upload.
Reboot the server to start using the new mods.