it's no secret that i am a fan of the 90s era of gaming, just look at this domain name. being a tech nerd, a gamer, and a data hoarder, makes for an interesting combination of interests that leads me down some wild collecting holes that for some reason still seems out of reach for a lot of computer users: console (and arcade) emulation. i largely blame this on the move of computing technology to transition to a mobile-first experience; emulation has always been somewhat of a monolith of an endeavor. right from the start, games are almost entirely downloaded from a public website, rather than played directly from within an app. beyond that, they come compressed in formats that are not often common on phones, and can be further confusing when you start dealing with games from the later generations released on optical media.
so, i will preface this guide with a very large disclaimer: if you're trying to follow this on a phone, look elsewhere. sure, follow this guide on a desktop and transfer the games to your phone when you're done. but if you're trying to amass a collection of games solely from your phone, you're in for a bad time.
so, what is the internet archive?
well, much like the home page says, "Internet Archive is a non-profit library of millions of free books, movies, software, music, websites, and more." you might have used the wayback machine at some point in time, trying to recover the contents of a web site that does not exist anymore. while that is one of the chief uses of the archive, it's also a DMCA-free haven, as it is registered as an actual library and all of its contents uploaded are seemingly free of the clutches of copyright lawyers. or at least that is my impression. i am not a lawyer. i just enjoy digital preservation and old video games.
so how do i find these old roms?
to do that, you'll need a bit of a primer on how collections of roms are organized, which will make your life infinitely easier when searching.
if you are interested in cartridge based systems, you only need to worry about one query: no-intro. this is a group of digital dumping enthusiasts who maintain databases of every console system and the status of every dumped game.
locating the no-intro sets could not be much easier, searching "no-intro 2021" gives us what are very recent (as of this writing) collections of many many console systems, available as full zip file packs. there won't be a single pack over a couple gigabytes in size so they are easy to consume as individual downloads. unpacking the zips will reveal thousands upon thousands of titles; which would take you literal years to play through every one. but hey, there is the entire super nes library, at your fingertips. enjoy.
that's nice, but how about newer stuff?
so while yes it is technically impressive to fit an entire console library on a small thumbdrive, it is becoming less and less so as the years go on and video games have gotten more impressive, and as such larger in size. so if your quest is to collect games from optical based systems, be it on something like a CD or DVD, the redump project has taken the helm at dumping and verifying optical media. this includes the sega dreamcast and its GD-ROM discs.
much like the no-intro sets on the internet archive, kind souls from across the internet have donated their time and bandwidth to upload full sets there as well. this is where you need to ask yourself a very important question.
do i care about archiving an entire console, or just playing a small set of games?
you need to be very honest with yourself when answering. the difference in collecting an entire set en masse, and grabbing just a couple games, greatly alters the amount of effort involved. but to start, the query "redump 2020" gets you quite the healthy list of systems. and you can always tack on something like "gamecube" or "playstation" to further whittle down your list.
once you found your relevant collection, downloading individual games is as simple as clicking the "SHOW ALL" link on the right side. you'll get a nice indexed list of every file, one by one, that you can just click and download. easy enough.
ok i am a dedicated and certified data hoarder, i want it all.
listen, you don't get here without being a bit of a nerd. so we're going to be nerds and download stuff in one shot via the command line.
install python. version 3.
create an empty directory where you want to store all your downloaded stuff. put this file in there. then launch a shell (or command prompt) in that directory where you placed the file, and run
python ia. you'll get a list of all the options it supports.
if you want to download an entire collection at once, it's as simple as giving it a single command.
python ia download $NAME_OF_SET where
$NAME_OF_SET is the endpoint URI of the collection, after the
archive.org/details/ part. so something like
python ia download GC_US_Arquivista would download the latest gamecube set, all in one shot. just don't close the window. us linux gurus know how to background this process with something like
the command line still scares me. i'm also quite a bit lazy.
fine. use the downthemall extension (available for either chrome, or firefox) on the neatly indexed pages (that SHOW ALL link) that list every game download. i'd just recommend setting it to download one at a time. it's actually more unreliable to try and queue up multiple at once.
the internet archive is slow / unreliable.
no it isn't. you're just impatient.
these sets are often times hundreds of gigabytes in size. even a modest internet connection, one that is 100 megabits, would still take you a signifcant amount of time to transfer everything. to offer some perspective, download 500GB of data would still take 12 hours if you saturated the connection. the internet archive doesn't offer nearly that fast of a speed to end users naturally, as bandwidth is finite. but, what it lacks in bandwidth it still offers up reliability. and when you are merely archiving, time is something you have a lot of.
on average i get around 15-20mbit downloads from the archive, which is by no means "fast" but it also is not terrible, either. downloading a full set will take days, yes. but it's a one-shot deal. the tradeoff in my opinion is well worth it, i am less concerned with 'instant gratification' of fast downloads and am just happy i have a reliable source to gorge in my data hoarding habits.