Fallout 4 configuration tool is an excellent free tool that makes it much more straight-forward to modfiy the game's ini file. You don't need the tool to complete this guide, however I recommend it if you have a lower-powered system and are regularly tweaking stuff. You can get it here[www.nexusmods.com].

Note: This guide WILL feature instructions for people not using this tool.

For this to work, you'll need to get the upscale mod, a super easy mod to install.

To run this mod you'll need the DLL Launcher mod. Download the Native Mod DLL launcher here[www.nexusmods.com]. After unzipping, you should find a d3d11.dll file, which you should just place in your Fallout 4 directory (steamapps\common\Fallout 4\, i.e. the same folder as the executable). If you're using a vanilla install, you shouldn't have to replace anything, but some other mod launchers, such as ENB, use this file. There are instructions on the Nexus page for DLL Launcher to fix this.

Now that you have the launcher installed, you'll need to get the Upscale mod itself. Download it from here[www.nexusmods.com], then unzip it and place the upscale folder into a new folder in the Fallout 4 directory named NativeMods. So, you should have a folder at ...\Steam\steamapps\common\Fallout 4\NativeMods\upscale, then in the upscale folder you should have an upscale.dll file.
If you've done everything right, you should be able to replicate this screenshot;

You should now have a d3d11.dll in the same folder as Fallout4.exe, as well as a NativeMods folder with an upscale folder inside, which should contain the upscale.dll.

With Fallout 4 Configuration Tool
Open the configuration tool, then click the "Tweaks" tab at the top. There should be a Custom Resolution section on the bottom right. Make sure Custom Resolution is enabled (should have a blue box, toggle by clicking on it). Now, set your X and Y values to the resolution you want. If you are unsure, see below for a list of 16:9 resolutions.

Manually
Go to your Documents folder, then go into My Games, then Fallout4. Now open Fallout4Prefs.ini with any text editor (notepad works). Now, press Ctrl+F and type iSize. You should be able to find two lines of code that say something like: and: . Set these values to what you want your resolution to be, with W being the number of horizontal pixels (X axis) and H being the vertical pixels (Y axis). If you are unsure, look below for a list of 16:9 resolutions.

Standard 16:9 resolutions

As an example, this is what it should look like in the tool if you are using 1280x720;
And this is what that same example should look like if you are editing the ini.

DON'T START THE GAME YET!
There are still a few settings we need to change to make it all work.

With Fallout 4 Configuration Tool
Under Custom Resolution, make sure Borderless is enabled, but Fullscreen, Maximize, and TopMost are all disabled. Items are enabled if they are blue, and disabled if they are black (very dark gray).
It should now look like this;

Save these settings with the button in the bottom left. If it comes up with a warning about plugins not being accepted, press No and then disable Warning for Invalidate Archives on the Settings tab of the tool.

Manually
Near the lines in Fallout4Prefs.ini that we edited earlier, you should find these lines;
Set bTopMostWindow, bMaximizeWindow, and bFull Screen to 0, and set bBorderless to 1.

Save this file.

You should now be able to run at a lower resolution than native. I hope you found this guide useful because I took ages to figure this out. It is basically just installing and setting up a mod, but with all the so called fix guides and videos out there it was difficult to find. Good luck. If you have any issues, please leave a comment.

There is an issue with this solution I have noticed where the dark green (default color) background of some UI elements is slightly shifted from where it should be. I don't know how to fix this, sorry, but it doesn't happen as noticably when you're ingame, so you should be able to get used to it pretty easilly.

Also, this will look more pixelated than regular fullscreen, as regular fullscreen typically has to share multiple pixels for each bit and can look more muddy. Generally this blurryness actually looks better, but there is not much to be done about it.