My Steam is in Swedish, so I apologize if you can't understand what the different options mean, but they are always in the same place in the lists, so just click on the same option I click and you will get it right. I will try my best to guess what the options are called in English.

Here are some things you should know before you start following my steps:
Illustration dimensions: 300x450 or 600x900 (px)
Banner dimensions: 1920x620 or 3840x1240 (px)
Logo dimensions: Not as important, but preferably 640x360 or 1280x720 (px)

Illustrations are the images you see in the home page of your library.
 
To change it, right click the image you want to change, then hover over Manage, then Choose custom illustration. This will open up your files, and you can choose whichever image file you want to use.

The logo is the little image that is usually located in the bottom left after you click on a game in the library. To change it, right click the background image and choose Choose custom logo. This will open your files and you can choose whichever image file you want to use.
 

The banner is the big image that is shown at the top of the screen after you have clicked on the game in your library. To change this, do the same as when you changed the logo, but click on the other option.
 

I own GTA V on Epic Games because I got it for free, but I added it to my Steam library (click here to see how) and changed the illustration, logo, and banner because non-Steam games have black and gray colors only, which is boring to look at.
Here is what I did:
Illustration (highlighted with blue/green rectangle):
 
Banner and logo:

I hope this guide was helpful, and I hope you can now make your Steam library more personal. This works on both Steam games and non-Steam games, but I personally only do it on my non-Steam games.

https://steamproxy.net/sharedfiles/filedetails/?id=2370120676