Firstly, make sure that you are logged into your Pterodactyl Panel.
Once you have downloaded the Jar file/mod pack that you want to use, you need to connect to your servers SFTP server, you can find these details here;
Upload your custom Jar file into the servers home directory (once logged in, just click Upload)
Once uploaded, you need to instruct your server to use your new .JAR file, to do this select "Configuration" > "Startup Parameters";
Under the "Server Jar File" section enter the name of your newly uploaded JAR file.
You can ignore the section that states your servers version, this is no longer relevant.
You can now go back to your "Console" and start your server is normal.
Congratulations, you are now using your custom Jar file!