I would format ALL USB flash drives to HFS+ (Mac OS extended with journaling enabled, GUID partition format). The only flash drives that SHOULD NOT be HFS+ are those that are also used to connect to Windows PCs.
The "photoslibrary" icon is not really "a file" -- rather, it's a "package" (I believe), or rather, a folder full of different items.
Try RIGHT clicking on it, and choose "show package contents" from the contextual menu that appears. Just look, don't mess with anything that's "in there".
Having said that, I don't see any problem with copying it to a flash drive.
You can connect that to the "target Mac" (your mom's new Mac), and "import" from there if you want.
BUT BE CAREFUL
If you try to copy the entire kit-and-kaboodle photoslibrary into your mom's existing home/pictures folder, there could be trouble with permissions.
It can be done, but precautions must be taken:
a. connect flash drive
b. DO NOT open the drive icon yet
c. click ONE TIME on the icon to select it
d. bring up the "get info" box for it (command-i)
e. at the bottom of get info, click the lock and enter the administrative password for your mom's NEW account on the NEW Mac
f. put a check into "ignore ownership on this volume" (sharing and permissions)
g. close get info.
(the above exercise will over-ride permissions issues with files copied from the drive)
You can now copy the entire photoslibrary icon into your mom's "home/pictures" folder, REPLACING THE FILE ALREADY IN THERE.
But realize if you do this, the old library is GONE.
Finally, you'll have to "tell photos" to open the new library if it can't find it.
Hold down the option key and open Photos, it will ask you to point the way to the new library.
WARNING:
I do not use (have NEVER used) iCloud for photos, I have no idea how any of the above works where iCloud storage is involved...