One good principle of data structure design is to ensure that every attribute of a table or object be subject to a well-understood constraint. This is important because if you or your program can count on valid data in the database, you are less likely to have program defects caused by bad data. You also spend less time writing code to handle error conditions, and you are more likely to write error-handling code up front.
In many cases these constraints can be defined at compile-time, in which case you can write a filter to ensure that the attribute always falls within range, or the attempt to save the attribute fails.
However, in many cases these constraints can change at run-time. For example, you may have a "cars" table that has "colour" as an attribute which initially takes on the values, say, of "red", "green" and "blue". It is possible during the execution of the program to add valid colours to that initial list, and new "cars" added may take on any colour in the up-to-date list of colours. Furthermore, you usually want this updated list of colours to survive a program restart.
To answer your question, it turns out that if you have a requirement for data constraint that can change at run-time, and those changes must survive a program restart, foreign keys are the simplest and most concise solution to the problem. The development cost is the addition of one table (e.g. "colours", a foreign key constraint to the "cars" table, and an index), and the run-time cost is the extra table lookup for the up-to-date colours to validate the data, and this run-time cost is usually mitigated by indexing and caching.
If you don't use foreign keys for these requirements, you must write software to manage the list, look valid entries, save it to disk, structure the data efficiently if the list is large, ensure that any updates to the list don't corrupt the list file, provide serial access to the list in case there are multiple readers and/or writers, and so on. i.e. You need to implement a lot of RDBMS functionality.