The OP's problems illustrate why -- before a major upgrade from one version of the OS to the next -- it's IMPERATIVE to create a fully-bootable cloned version of your "older install" using either CarbonCopyCloner or SuperDuper.
IF the OP had such a backup, it would be "child's play" to "get back to where he once belonged".
But -- WITHOUT a cloned backup -- getting back ain't gonna be easy.
What you might try -- this is going to be A LOT OF WORK.
You'll need an EXTERNAL drive. You CANNOT DO THIS on Low Sierra.
Install a copy of High Sierra onto the external drive. Yes, it's going to take time.
Put the Photos Library (the one that won't open) into your "home/pictures" folder on the external drive, REPLACING whatever is installed onto it.
Now... you ought to be able to reopen the library (since HS is running).
Next, select ALL photos and choose to export them to a folder (I have no idea how many pics you have in there). You'll probably need another drive or USB flash drive.
Now, boot back to Low Sierra.
On your Low Sierra (internal) drive, open home/pictures.
RIGHT CLICK on the Photos library and choose "show package contents"
It will open and show you the contents of the Photos Library.
See the folder named "Masters"? Open it.
Now type "command-a" (select all) and select everything inside the Masters folder.
Move this to the trash, and empty it.
With the previous contents removed, now open Photos.
Choose to import the folder with the previously-exported pics from the HS install.
Does Photos import them?
Again -- this sounds like a lot of work, because it is.
You'll have to go through this, because you DIDN'T take the important step I described in the first paragraph above BEFORE you upgraded from Low Sierra to High Sierra...
